Jul 15 13:51:17.404896 ip-10-0-129-208 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.804664    2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807911    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807924    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807929    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807933    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807937    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807942    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:51:17.897101 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807946    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:51:17.876436 ip-10-0-129-208 systemd[1]: Started Kubernetes Kubelet.
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807949    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807957    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807961    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807964    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807967    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807970    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807973    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807975    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807978    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807981    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807985    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807987    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807991    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807994    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.807997    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808001    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808003    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808006    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808008    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:51:17.898439 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808011    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808013    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808016    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808019    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808021    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808024    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808026    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808028    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808031    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808033    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808036    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808038    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808041    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808043    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808046    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808049    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808052    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808055    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808057    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:51:17.899570 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808060    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808063    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808065    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808068    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808070    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808073    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808076    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808078    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808081    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808083    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808086    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808088    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808091    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808093    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808096    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808098    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808100    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808103    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808105    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808108    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:51:17.900712 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808112    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808115    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808118    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808121    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808123    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808126    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808128    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808131    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808134    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808137    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808139    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808141    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808144    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808146    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808149    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808151    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808154    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808156    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808160    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808162    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:51:17.901693 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808165    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808168    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808170    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808173    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808176    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808178    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808180    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808183    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808186    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808188    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808191    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808193    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808195    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808198    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808922    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808929    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808932    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808936    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808939    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:51:18.120575 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808942    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808945    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808948    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808957    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808960    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808963    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808965    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808968    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808971    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808974    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808976    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808980    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808983    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808987    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808991    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808994    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808997    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.808999    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809002    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:51:18.138878 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809004    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809007    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809009    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809012    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809014    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809017    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809019    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809022    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809024    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809027    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809030    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809032    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809035    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809037    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809040    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809042    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809045    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809054    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809057    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809060    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:51:18.139469 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809063    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809066    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809069    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809073    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809075    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809078    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809081    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809084    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809086    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809089    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809091    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809095    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809097    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809100    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809102    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809106    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809109    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809112    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809115    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:51:18.140105 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809117    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809120    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809122    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809124    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809127    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809129    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809131    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809134    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809137    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809140    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809142    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809152    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809154    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809157    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809163    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809165    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809168    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809171    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809173    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809176    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809179    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:51:18.140685 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809181    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809184    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809186    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809188    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809191    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809194    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809196    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809199    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809201    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809204    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809206    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809209    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809211    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809214    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809216    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809661    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809666    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809668    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809671    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809674    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:51:18.141322 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809676    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809679    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809688    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809691    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809693    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809697    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809700    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809702    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809705    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809708    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809710    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809713    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809716    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809720    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809722    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809725    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809727    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809730    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809733    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809735    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:51:18.141908 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809738    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809740    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809742    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809745    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809747    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809750    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809752    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809755    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809757    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809760    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809762    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809765    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809767    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809770    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809772    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809780    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809783    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809787    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809790    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809792    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:51:18.142501 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809795    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809797    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809800    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809802    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809805    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809809    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809811    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809814    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809823    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809825    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809828    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809830    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809835    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809838    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809841    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809845    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809849    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809852    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809854    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:51:18.143051 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809858    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809863    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809866    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809869    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809872    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809874    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809877    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809879    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809882    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809890    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809894    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809896    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809899    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809901    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809904    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809907    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809909    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809911    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809915    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809917    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:51:18.143642 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809920    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809922    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809925    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809927    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809930    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809932    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809935    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809937    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809939    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809942    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809944    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809947    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809949    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809952    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.809954    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810023    2573 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810030    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810042    2573 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810046    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810058    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810061    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 13:51:18.144273 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810065    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810077    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810080    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810083    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810087    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810090    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810093    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810096    2573 flags.go:64] FLAG: --cgroup-root=""
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810099    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810103    2573 flags.go:64] FLAG: --client-ca-file=""
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810106    2573 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810109    2573 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810115    2573 flags.go:64] FLAG: --cluster-domain=""
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810118    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810121    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810124    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810127    2573 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810131    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810134    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810137    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810140    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810143    2573 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810146    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810149    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 13:51:18.144910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810152    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810155    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810159    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810162    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810165    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810167    2573 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810170    2573 flags.go:64] FLAG: --enable-server="true"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810173    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810180    2573 flags.go:64] FLAG: --event-burst="100"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810185    2573 flags.go:64] FLAG: --event-qps="50"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810193    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810196    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810199    2573 flags.go:64] FLAG: --eviction-hard=""
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810203    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810206    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810209    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810212    2573 flags.go:64] FLAG: --eviction-soft=""
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810216    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810219    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810222    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810224    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810227    2573 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810230    2573 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810233    2573 flags.go:64] FLAG: --feature-gates=""
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810236    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 13:51:18.145551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810239    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810242    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810245    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810248    2573 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810251    2573 flags.go:64] FLAG: --help="false"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810254    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-129-208.ec2.internal"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810257    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810260    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810263    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810266    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810269    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810272    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810275    2573 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810277    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810280    2573 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810283    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810287    2573 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810290    2573 flags.go:64] FLAG: --kube-reserved=""
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810298    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810301    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810304    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810307    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810310    2573 flags.go:64] FLAG: --lock-file=""
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810313    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 13:51:18.146299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810317    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810320    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810325    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810328    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810331    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810334    2573 flags.go:64] FLAG: --logging-format="text"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810337    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810340    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810342    2573 flags.go:64] FLAG: --manifest-url=""
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810345    2573 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810349    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810352    2573 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810356    2573 flags.go:64] FLAG: --max-pods="110"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810359    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810362    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810365    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810368    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810371    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810373    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810376    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810383    2573 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810386    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810394    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 13:51:18.146944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810397    2573 flags.go:64] FLAG: --pod-cidr=""
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810402    2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810410    2573 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810413    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.810456    2573 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.811442    2573 flags.go:64] FLAG: --port="10250"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.811489    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.811497    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-055ba2b5b915515e2"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.811504    2573 flags.go:64] FLAG: --qos-reserved=""
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.811511    2573 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.811518    2573 flags.go:64] FLAG: --register-node="true"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812059    2573 flags.go:64] FLAG: --register-with-taints=""
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812173    2573 flags.go:64] FLAG: --registry-burst="10"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812179    2573 flags.go:64] FLAG: --registry-qps="5"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812183    2573 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812187    2573 flags.go:64] FLAG: --reserved-memory=""
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812192    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812195    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812198    2573 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812201    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812205    2573 flags.go:64] FLAG: --runonce="false"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812208    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812211    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812214    2573 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812217    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 13:51:18.147725 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812220    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812223    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812226    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812230    2573 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812232    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812235    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812238    2573 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812241    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812244    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812247    2573 flags.go:64] FLAG: --system-cgroups=""
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812250    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812259    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812262    2573 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812266    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812272    2573 flags.go:64] FLAG: --tls-min-version=""
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812275    2573 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812277    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812280    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812283    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812299    2573 flags.go:64] FLAG: --v="2"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812304    2573 flags.go:64] FLAG: --version="false"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812307    2573 flags.go:64] FLAG: --vmodule=""
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812311    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812315    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812436    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:51:18.148438 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812440    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812444    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812447    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812450    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812453    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812455    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812459    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812462    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812464    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812469    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812473    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812476    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812479    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812483    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812486    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812489    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812492    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812498    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812501    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:51:18.151947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812503    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812506    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812508    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812511    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812514    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812516    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812519    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812522    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812524    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812528    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812531    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812533    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812536    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812538    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812541    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812543    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812546    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812548    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812551    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812553    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:51:18.152745 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812556    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812558    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812561    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812564    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812566    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812568    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812571    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812574    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812576    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812579    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812583    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812585    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812588    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812590    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812593    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812595    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812597    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812600    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812620    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812649    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:51:18.153340 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812656    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812660    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812663    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812667    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812671    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812674    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812678    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812680    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812683    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812686    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812688    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812691    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812694    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812696    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812699    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812701    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812704    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812706    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812709    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812712    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:51:18.154515 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812714    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812717    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812719    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812722    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812725    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812728    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812730    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812733    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812735    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812738    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812741    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812744    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812747    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812751    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812755    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812759    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812761    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812764    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:51:18.155182 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.812767    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.812773    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.819384    2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.819399    2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.820176    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.820183    2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820241    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820246    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820249    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820252    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820255    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820259    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820262    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820265    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820267    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:51:19.750078 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820270    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820273    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820275    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820278    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820281    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820283    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820286    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820288    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820291    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820293    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820296    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820301    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820304    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820307    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820310    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820313    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820316    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820318    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820321    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:51:19.751199 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820324    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820327    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820330    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820332    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820335    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820337    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820340    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820342    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820345    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820347    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820350    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820353    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820355    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820358    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820360    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820363    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820366    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820368    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820371    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820373    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:51:19.751826 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820376    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820378    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820381    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820383    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820386    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820389    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820391    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820394    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820397    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820400    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820402    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820405    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820408    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820411    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820414    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820416    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820420    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820422    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820425    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820427    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:51:19.752443 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820430    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820434    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820437    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820442    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820445    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820448    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820450    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820453    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820456    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820459    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820463    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820479    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820483    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820486    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820489    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820491    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820494    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820497    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820499    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:51:19.752981 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820502    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820505    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820510    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820512    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820515    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820518    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820520    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820523    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820527    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820530    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820532    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820535    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.820539    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820689    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820694    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:51:19.753534 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820697    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820701    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820704    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820706    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820709    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820711    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820714    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820716    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820719    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820721    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820724    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820727    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820729    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820732    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820734    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820737    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820740    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820742    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820745    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820748    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:51:19.753928 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820753    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820756    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820759    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820761    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820764    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820766    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820769    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820772    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820774    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820777    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820779    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820782    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820784    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820788    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820790    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820793    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820795    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820798    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820800    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820803    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:51:19.754549 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820805    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820808    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820810    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820813    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820815    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820817    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820820    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820822    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820825    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820827    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820830    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820833    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820836    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820839    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820842    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820844    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820847    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820849    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820853    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:51:19.755067 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820861    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820864    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820866    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820869    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820871    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820874    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820878    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820881    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820883    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820886    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820888    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820891    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820893    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820896    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820898    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820901    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820903    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820905    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820908    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820911    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820913    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:51:19.755601 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820916    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820918    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820921    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820923    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820927    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820929    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820932    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820934    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820937    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820941    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820944    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820948    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820952    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820955    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820958    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820961    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:51:19.756163 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:17.820964    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.820970    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.821100    2573 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.824039    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.825223    2573 server.go:1019] "Starting client certificate rotation"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.825349    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.825378    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.827649    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.831930    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.843325    2573 log.go:25] "Validated CRI v1 runtime API"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.845231    2573 log.go:25] "Validated CRI v1 image API"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.846068    2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 13:51:19.756628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.849103    2573 fs.go:135] Filesystem UUIDs: map[241b9799-5ec3-412b-a39d-971fa6a0f085:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ad1f3299-c898-4658-b104-c731191ffe86:/dev/nvme0n1p3]
Jul 15 13:51:19.756966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.849117    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 13:51:19.757068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.854454    2573 manager.go:216] Machine: {Timestamp:2026-07-15 13:51:17.852622247 +0000 UTC m=+0.329155518 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3204913 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23c0611295a2ea24392cd94b429e32 SystemUUID:ec23c061-1295-a2ea-2439-2cd94b429e32 BootID:98d7d266-a2cb-4a73-98dd-2a25afa23e42 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8e:87:0d:b3:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8e:87:0d:b3:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:7e:e2:f6:70:c3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 13:51:19.757068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.854558    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 13:51:19.757068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.854654    2573 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 13:51:19.757068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.855675    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 13:51:19.757068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.857703    2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 13:51:19.757068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.857735    2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-208.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 13:51:19.757068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.858869    2573 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.858879    2573 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.858887    2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.859486    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.860439    2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.862707    2573 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.862827    2573 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.862855    2573 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.862866    2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.862888    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.862898    2573 kubelet.go:398] "Adding apiserver pod source"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.862907    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.863297    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.863313    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.866518    2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.868344    2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.868387    2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.868594    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.868625    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.868634    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.868643    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.868653    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.868661    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.869061    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.869070    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.869078    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.869088    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.869102    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.869114    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.870555    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.870565    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.874502    2573 server.go:1268] "Started kubelet"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.874616    2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.874715    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.874778    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.875360    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-208.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.875898    2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.875971    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-208.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.875988    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 13:51:19.757307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.877905    2573 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.880859    2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.882541    2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.882546    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.886344    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.886348    2573 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.886376    2573 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.887648    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.887756    2573 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.887764    2573 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.887874    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mzppr"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.887004    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-208.ec2.internal.18c27ad9b6246da0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-208.ec2.internal,UID:ip-10-0-129-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-208.ec2.internal,},FirstTimestamp:2026-07-15 13:51:17.874470304 +0000 UTC m=+0.351003575,LastTimestamp:2026-07-15 13:51:17.874470304 +0000 UTC m=+0.351003575,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-208.ec2.internal,}"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.889016    2573 factory.go:153] Registering CRI-O factory
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.889035    2573 factory.go:223] Registration of the crio container factory successfully
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.889090    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.889099    2573 factory.go:55] Registering systemd factory
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.889108    2573 factory.go:223] Registration of the systemd container factory successfully
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.889131    2573 factory.go:103] Registering Raw factory
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.889145    2573 manager.go:1196] Started watching for new ooms in manager
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.889900    2573 manager.go:349] Starting recovery of all containers
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.894101    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.894109    2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-208.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.895075    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mzppr"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.901139    2573 manager.go:354] Recovery completed
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.905204    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.909125    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.909154    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.909167    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.909719    2573 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.909732    2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.909739    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.909749    2573 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 13:51:19.758318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.910465    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.910487    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.910501    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.910651    2573 policy_none.go:49] "None policy: Start"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.910666    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.910679    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.911408    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-208.ec2.internal.18c27ad9b83568af  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-208.ec2.internal,UID:ip-10-0-129-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-208.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-208.ec2.internal,},FirstTimestamp:2026-07-15 13:51:17.909137583 +0000 UTC m=+0.385670853,LastTimestamp:2026-07-15 13:51:17.909137583 +0000 UTC m=+0.385670853,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-208.ec2.internal,}"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.911483    2573 policy_none.go:47] "Start"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.961693    2573 manager.go:337] "Starting Device Plugin manager"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.961724    2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.961736    2573 server.go:87] "Starting device plugin registration server"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.962006    2573 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.962018    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.962095    2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.962186    2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:17.962196    2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.962794    2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:17.962850    2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.042038    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.043172    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.043208    2573 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.043228    2573 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:18.043259    2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.045920    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.062964    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.064115    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.064140    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.064153    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.064180    2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.074271    2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:18.074291    2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-208.ec2.internal\": node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:18.089335    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.144080    2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal"]
Jul 15 13:51:19.759162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.144196    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.145750    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.145779    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.145791    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.148162    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.148276    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.148303    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.148967    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.148967    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.149019    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.149038    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.148992    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.149064    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.151824    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.151849    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.152579    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.152618    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:18.152641    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:18.184398    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-208.ec2.internal\" not found" node="ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:19.545487    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-208.ec2.internal\" not found" node="ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:19.546469    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.546522    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e43c7d6073f1b1b5cc49a22829a91c6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal\" (UID: \"e43c7d6073f1b1b5cc49a22829a91c6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.546544    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e43c7d6073f1b1b5cc49a22829a91c6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal\" (UID: \"e43c7d6073f1b1b5cc49a22829a91c6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.546562    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e182e8671520a5dbfd171d137a66aa6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-208.ec2.internal\" (UID: \"5e182e8671520a5dbfd171d137a66aa6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:19.646517    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:19.760042 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.647508    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e43c7d6073f1b1b5cc49a22829a91c6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal\" (UID: \"e43c7d6073f1b1b5cc49a22829a91c6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760706 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.647551    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e43c7d6073f1b1b5cc49a22829a91c6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal\" (UID: \"e43c7d6073f1b1b5cc49a22829a91c6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760706 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.647600    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e182e8671520a5dbfd171d137a66aa6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-208.ec2.internal\" (UID: \"5e182e8671520a5dbfd171d137a66aa6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760706 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.647635    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e182e8671520a5dbfd171d137a66aa6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-208.ec2.internal\" (UID: \"5e182e8671520a5dbfd171d137a66aa6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760706 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.647513    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e43c7d6073f1b1b5cc49a22829a91c6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal\" (UID: \"e43c7d6073f1b1b5cc49a22829a91c6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760706 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.647645    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e43c7d6073f1b1b5cc49a22829a91c6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal\" (UID: \"e43c7d6073f1b1b5cc49a22829a91c6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.760706 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:19.747689    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:19.979514 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.843402    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.979514 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:19.848106    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:19.979514 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:19.848203    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:19.979514 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:19.948599    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:20.086880 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.049043    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-208.ec2.internal\" not found"
Jul 15 13:51:20.086880 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.072045    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:51:20.200162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.144700    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:20.200162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.158362    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 13:51:20.200162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.160030    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal"
Jul 15 13:51:20.200162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.167794    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 13:51:20.200162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.181862    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 13:51:20.200162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.181975    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 13:51:20.200162 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.182006    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.220186    2573 apiserver.go:52] "Watching apiserver"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.239301    2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.239389    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.240303    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-pnhlx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g","openshift-image-registry/node-ca-qfvwg","openshift-multus/multus-additional-cni-plugins-vffsz","openshift-multus/network-metrics-daemon-xh2hs","openshift-network-diagnostics/network-check-target-zbg6k","openshift-ovn-kubernetes/ovnkube-node-mwc8d","kube-system/global-pull-secret-syncer-qnhlv","kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qjc4g","openshift-dns/node-resolver-k5mg8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal","openshift-multus/multus-c5qpr","openshift-network-operator/iptables-alerter-qt2vd"]
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.242767    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.244904    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.246363    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.246372    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.246368    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.246377    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.246410    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.246414    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.246424    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2sn6p\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.247121    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.247304    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249412    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249437    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249605    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249682    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249777    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-systemd\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249804    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-ovn\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249831    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-env-overrides\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.320015 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249856    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-system-cni-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249886    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-cni-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249909    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-socket-dir-parent\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249925    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-log-socket\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249941    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-slash\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249956    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-run-ovn-kubernetes\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249969    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-cni-bin\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.249992    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovn-node-metrics-cert\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250007    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-os-release\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250022    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-kubelet\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250035    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-systemd-units\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250048    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250065    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250080    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovnkube-config\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322012 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250093    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-cnibin\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250110    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-conf-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250126    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-daemon-config\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250143    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovnkube-script-lib\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250157    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-multus-certs\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250170    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-node-log\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250183    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-k8s-cni-cncf-io\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250209    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-netns\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250231    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptxfx\" (UniqueName: \"kubernetes.io/projected/4d016d94-23a8-4b4f-91f8-acd413e4558e-kube-api-access-ptxfx\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250247    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-kubelet\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250261    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-run-netns\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250276    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-var-lib-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250289    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-etc-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250303    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9mkn\" (UniqueName: \"kubernetes.io/projected/2520d340-bfc2-4022-ad9b-4956b7a7b028-kube-api-access-j9mkn\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.322421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250316    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-cni-multus\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250328    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-etc-kubernetes\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250356    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-cni-bin\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250369    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-hostroot\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250383    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-cni-netd\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250397    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d016d94-23a8-4b4f-91f8-acd413e4558e-cni-binary-copy\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250671    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250687    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.250705    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x6xmf\""
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.251349    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.251665    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-948x6\""
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.252537    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.252639    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.252754    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r68pz\""
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.254518    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.254587    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.254732    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 13:46:19 +0000 UTC" deadline="2028-03-08 18:59:31.04794632 +0000 UTC"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.254758    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14453h8m10.793189705s"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.256808    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.256858    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.256860    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.259006    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.323631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.261248    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.262692    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.262779    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.262812    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.262894    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2h4ts\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.263484    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.263818    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.263825    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q6szc\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.263975    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.265702    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.265812    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.265868    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.265899    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.265910    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7rqvj\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.268021    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.268364    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.268519    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.268611    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2k4vz\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.270257    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.270345    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.270804    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.270882    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.271351    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6wtwg\""
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.278792    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vztth"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.287019    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vztth"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.306477    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:51:20.330539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.308136    2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.343943    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350698    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-kubernetes\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350730    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtm4z\" (UniqueName: \"kubernetes.io/projected/0d451b46-14cc-475e-9d9a-64dc6143af58-kube-api-access-qtm4z\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350752    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350793    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-device-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350817    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwtqj\" (UniqueName: \"kubernetes.io/projected/794fea14-734a-4b7c-a471-a7cf6c26615b-kube-api-access-gwtqj\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350835    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbzss\" (UniqueName: \"kubernetes.io/projected/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-kube-api-access-nbzss\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350860    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-ovn\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350876    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-system-cni-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350890    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-socket-dir-parent\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350912    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysctl-d\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350934    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-var-lib-kubelet\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350947    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-ovn\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.436600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350955    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-socket-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350956    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-system-cni-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350968    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-socket-dir-parent\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.350993    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-etc-selinux\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351016    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-tmp-dir\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351038    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-cni-bin\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351060    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovn-node-metrics-cert\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351080    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-kubelet\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351101    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-lib-modules\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351111    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-cni-bin\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351121    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-system-cni-dir\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-systemd-units\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351146    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-kubelet\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351205    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovnkube-config\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.437384 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351211    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-systemd-units\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351235    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-conf-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351322    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-daemon-config\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351359    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-conf-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351385    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6bwl\" (UniqueName: \"kubernetes.io/projected/f40d9221-15b5-4836-83ad-42bdad956a48-kube-api-access-p6bwl\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351403    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351423    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovnkube-script-lib\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351445    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-multus-certs\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351460    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysconfig\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351474    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysctl-conf\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351486    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-run\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351508    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6edabe15-150f-4758-8b05-a9c0f1b266bf-dbus\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351520    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-multus-certs\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351531    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cnibin\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.439401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351559    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-netns\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351583    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af8f200d-6b67-4f66-8c9a-2d9e423b9247-konnectivity-ca\") pod \"konnectivity-agent-pnhlx\" (UID: \"af8f200d-6b67-4f66-8c9a-2d9e423b9247\") " pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351599    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-modprobe-d\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351610    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-netns\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351621    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-registration-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351646    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6edabe15-150f-4758-8b05-a9c0f1b266bf-kubelet-config\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351667    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351685    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-var-lib-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351708    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-var-lib-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351732    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-etc-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351752    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9mkn\" (UniqueName: \"kubernetes.io/projected/2520d340-bfc2-4022-ad9b-4956b7a7b028-kube-api-access-j9mkn\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351768    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-etc-kubernetes\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351782    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af8f200d-6b67-4f66-8c9a-2d9e423b9247-agent-certs\") pod \"konnectivity-agent-pnhlx\" (UID: \"af8f200d-6b67-4f66-8c9a-2d9e423b9247\") " pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351796    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba09580d-4c2a-452e-b857-c3c30f6d3593-tmp\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.440174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351816    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-sys-fs\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351832    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351833    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-etc-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351851    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d016d94-23a8-4b4f-91f8-acd413e4558e-cni-binary-copy\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351896    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-etc-kubernetes\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351897    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d451b46-14cc-475e-9d9a-64dc6143af58-host-slash\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351940    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351963    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-os-release\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.351986    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352004    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-systemd\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352025    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-env-overrides\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352029    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-systemd\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352048    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-cni-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352066    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkl8k\" (UniqueName: \"kubernetes.io/projected/ba09580d-4c2a-452e-b857-c3c30f6d3593-kube-api-access-vkl8k\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.440988 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352081    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d451b46-14cc-475e-9d9a-64dc6143af58-iptables-alerter-script\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352095    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cni-binary-copy\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352117    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-log-socket\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-slash\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352145    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-run-ovn-kubernetes\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352160    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-os-release\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352173    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352189    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352194    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-cni-dir\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352206    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-cnibin\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352224    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-slash\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352229    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352243    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f40d9221-15b5-4836-83ad-42bdad956a48-host\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352258    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22mf8\" (UniqueName: \"kubernetes.io/projected/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-kube-api-access-22mf8\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.441794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352265    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-log-socket\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352273    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hnml\" (UniqueName: \"kubernetes.io/projected/fbb22632-1c12-4275-a177-5dbc51f21162-kube-api-access-9hnml\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352289    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352329    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-cnibin\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352350    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-os-release\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352370    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-run-ovn-kubernetes\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352372    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-run-openvswitch\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352290    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-tuned\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352396    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-node-log\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352410    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-k8s-cni-cncf-io\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352431    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-run-k8s-cni-cncf-io\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352450    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-node-log\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352473    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptxfx\" (UniqueName: \"kubernetes.io/projected/4d016d94-23a8-4b4f-91f8-acd413e4558e-kube-api-access-ptxfx\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352488    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-host\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352502    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-kubelet\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352520    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-run-netns\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352534    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-cni-multus\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352556    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-cni-multus\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352576    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-kubelet\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352596    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-run-netns\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352682    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-systemd\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352705    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-sys\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352729    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-cni-bin\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352761    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-hostroot\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352800    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-hostroot\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352826    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-hosts-file\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352825    2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352826    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d016d94-23a8-4b4f-91f8-acd413e4558e-host-var-lib-cni-bin\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352842    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f40d9221-15b5-4836-83ad-42bdad956a48-serviceca\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352866    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-cni-netd\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.442972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.352887    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2520d340-bfc2-4022-ad9b-4956b7a7b028-host-cni-netd\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.443785 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.355013    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-env-overrides\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.443785 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.355041    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovnkube-config\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.443785 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.355100    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4d016d94-23a8-4b4f-91f8-acd413e4558e-multus-daemon-config\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.443785 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.355169    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d016d94-23a8-4b4f-91f8-acd413e4558e-cni-binary-copy\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.443785 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.355477    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovnkube-script-lib\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.443785 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.355843    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2520d340-bfc2-4022-ad9b-4956b7a7b028-ovn-node-metrics-cert\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.443785 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.362294    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptxfx\" (UniqueName: \"kubernetes.io/projected/4d016d94-23a8-4b4f-91f8-acd413e4558e-kube-api-access-ptxfx\") pod \"multus-c5qpr\" (UID: \"4d016d94-23a8-4b4f-91f8-acd413e4558e\") " pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:20.443785 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.362600    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9mkn\" (UniqueName: \"kubernetes.io/projected/2520d340-bfc2-4022-ad9b-4956b7a7b028-kube-api-access-j9mkn\") pod \"ovnkube-node-mwc8d\" (UID: \"2520d340-bfc2-4022-ad9b-4956b7a7b028\") " pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.453943    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-hosts-file\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.453969    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f40d9221-15b5-4836-83ad-42bdad956a48-serviceca\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.453993    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-kubernetes\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454046    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-kubernetes\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454085    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-hosts-file\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454096    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtm4z\" (UniqueName: \"kubernetes.io/projected/0d451b46-14cc-475e-9d9a-64dc6143af58-kube-api-access-qtm4z\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454124    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454139    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-device-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454154    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwtqj\" (UniqueName: \"kubernetes.io/projected/794fea14-734a-4b7c-a471-a7cf6c26615b-kube-api-access-gwtqj\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454168    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbzss\" (UniqueName: \"kubernetes.io/projected/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-kube-api-access-nbzss\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454188    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysctl-d\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454198    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454209    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-var-lib-kubelet\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.553354 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454221    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-device-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454235    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-socket-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454258    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-etc-selinux\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454261    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-var-lib-kubelet\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454282    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-tmp-dir\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454304    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-lib-modules\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454326    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-system-cni-dir\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454352    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysctl-d\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454369    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6bwl\" (UniqueName: \"kubernetes.io/projected/f40d9221-15b5-4836-83ad-42bdad956a48-kube-api-access-p6bwl\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454374    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-socket-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454391    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454403    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-etc-selinux\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454406    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-system-cni-dir\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454417    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysconfig\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454459    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysctl-conf\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454461    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysconfig\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454457    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-lib-modules\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454477    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-run\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454498    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6edabe15-150f-4758-8b05-a9c0f1b266bf-dbus\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454514    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cnibin\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454539    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af8f200d-6b67-4f66-8c9a-2d9e423b9247-konnectivity-ca\") pod \"konnectivity-agent-pnhlx\" (UID: \"af8f200d-6b67-4f66-8c9a-2d9e423b9247\") " pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454540    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-run\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454563    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-sysctl-conf\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454576    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6edabe15-150f-4758-8b05-a9c0f1b266bf-dbus\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454576    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cnibin\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454592    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-modprobe-d\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454609    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-registration-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454625    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6edabe15-150f-4758-8b05-a9c0f1b266bf-kubelet-config\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454641    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-modprobe-d\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.554966 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454646    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454655    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-registration-dir\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454664    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af8f200d-6b67-4f66-8c9a-2d9e423b9247-agent-certs\") pod \"konnectivity-agent-pnhlx\" (UID: \"af8f200d-6b67-4f66-8c9a-2d9e423b9247\") " pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454670    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6edabe15-150f-4758-8b05-a9c0f1b266bf-kubelet-config\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454681    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba09580d-4c2a-452e-b857-c3c30f6d3593-tmp\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454701    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-sys-fs\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454717    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454735    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d451b46-14cc-475e-9d9a-64dc6143af58-host-slash\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454751    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454756    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454777    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-os-release\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454789    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/794fea14-734a-4b7c-a471-a7cf6c26615b-sys-fs\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454800    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454816    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d451b46-14cc-475e-9d9a-64dc6143af58-host-slash\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.555830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454819    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkl8k\" (UniqueName: \"kubernetes.io/projected/ba09580d-4c2a-452e-b857-c3c30f6d3593-kube-api-access-vkl8k\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454835    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d451b46-14cc-475e-9d9a-64dc6143af58-iptables-alerter-script\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454875    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cni-binary-copy\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454910    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454925    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f40d9221-15b5-4836-83ad-42bdad956a48-host\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454943    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22mf8\" (UniqueName: \"kubernetes.io/projected/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-kube-api-access-22mf8\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454959    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hnml\" (UniqueName: \"kubernetes.io/projected/fbb22632-1c12-4275-a177-5dbc51f21162-kube-api-access-9hnml\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454975    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-tuned\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.454992    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-host\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455008    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-systemd\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455041    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-sys\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455094    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-sys\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455669    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f40d9221-15b5-4836-83ad-42bdad956a48-serviceca\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455755    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-tmp-dir\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.557447 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.454877    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.455878    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs podName:fbb22632-1c12-4275-a177-5dbc51f21162 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:20.955857653 +0000 UTC m=+2.075663219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs") pod "network-metrics-daemon-xh2hs" (UID: "fbb22632-1c12-4275-a177-5dbc51f21162") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455919    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455957    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af8f200d-6b67-4f66-8c9a-2d9e423b9247-konnectivity-ca\") pod \"konnectivity-agent-pnhlx\" (UID: \"af8f200d-6b67-4f66-8c9a-2d9e423b9247\") " pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455961    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.455978    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f40d9221-15b5-4836-83ad-42bdad956a48-host\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.456034    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-os-release\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.456047    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-host\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.456080    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-systemd\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.456101    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.456143    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret podName:6edabe15-150f-4758-8b05-a9c0f1b266bf nodeName:}" failed. No retries permitted until 2026-07-15 13:51:20.956129734 +0000 UTC m=+2.075935297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret") pod "global-pull-secret-syncer-qnhlv" (UID: "6edabe15-150f-4758-8b05-a9c0f1b266bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.456379    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d451b46-14cc-475e-9d9a-64dc6143af58-iptables-alerter-script\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.456428    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-cni-binary-copy\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:20.558315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.457807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af8f200d-6b67-4f66-8c9a-2d9e423b9247-agent-certs\") pod \"konnectivity-agent-pnhlx\" (UID: \"af8f200d-6b67-4f66-8c9a-2d9e423b9247\") " pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.457886    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba09580d-4c2a-452e-b857-c3c30f6d3593-tmp\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.458023    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba09580d-4c2a-452e-b857-c3c30f6d3593-etc-tuned\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.469627    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.469647    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.469659    2573 projected.go:196] Error preparing data for projected volume kube-api-access-z9whb for pod openshift-network-diagnostics/network-check-target-zbg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.469720    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb podName:7cee1a78-4f58-439f-8c26-66f336634a49 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:20.969706076 +0000 UTC m=+2.089511655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z9whb" (UniqueName: "kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb") pod "network-check-target-zbg6k" (UID: "7cee1a78-4f58-439f-8c26-66f336634a49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.471615    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbzss\" (UniqueName: \"kubernetes.io/projected/118b09fc-fa2f-4f5e-9418-19ad1aabe3f1-kube-api-access-nbzss\") pod \"node-resolver-k5mg8\" (UID: \"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1\") " pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.471949    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtm4z\" (UniqueName: \"kubernetes.io/projected/0d451b46-14cc-475e-9d9a-64dc6143af58-kube-api-access-qtm4z\") pod \"iptables-alerter-qt2vd\" (UID: \"0d451b46-14cc-475e-9d9a-64dc6143af58\") " pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.472169    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hnml\" (UniqueName: \"kubernetes.io/projected/fbb22632-1c12-4275-a177-5dbc51f21162-kube-api-access-9hnml\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.472248    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwtqj\" (UniqueName: \"kubernetes.io/projected/794fea14-734a-4b7c-a471-a7cf6c26615b-kube-api-access-gwtqj\") pod \"aws-ebs-csi-driver-node-6m27g\" (UID: \"794fea14-734a-4b7c-a471-a7cf6c26615b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.472299    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkl8k\" (UniqueName: \"kubernetes.io/projected/ba09580d-4c2a-452e-b857-c3c30f6d3593-kube-api-access-vkl8k\") pod \"tuned-qjc4g\" (UID: \"ba09580d-4c2a-452e-b857-c3c30f6d3593\") " pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.472413    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6bwl\" (UniqueName: \"kubernetes.io/projected/f40d9221-15b5-4836-83ad-42bdad956a48-kube-api-access-p6bwl\") pod \"node-ca-qfvwg\" (UID: \"f40d9221-15b5-4836-83ad-42bdad956a48\") " pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:20.560306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.472992    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22mf8\" (UniqueName: \"kubernetes.io/projected/b6e24913-c8a6-4c95-9a1c-e8755606b7ad-kube-api-access-22mf8\") pod \"multus-additional-cni-plugins-vffsz\" (UID: \"b6e24913-c8a6-4c95-9a1c-e8755606b7ad\") " pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:21.034779 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.958895    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:21.034779 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:20.958933    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:21.034779 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.959028    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:21.034779 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.959065    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:21.034779 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.959080    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret podName:6edabe15-150f-4758-8b05-a9c0f1b266bf nodeName:}" failed. No retries permitted until 2026-07-15 13:51:21.959066449 +0000 UTC m=+3.078872007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret") pod "global-pull-secret-syncer-qnhlv" (UID: "6edabe15-150f-4758-8b05-a9c0f1b266bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:21.034779 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:20.959194    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs podName:fbb22632-1c12-4275-a177-5dbc51f21162 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:21.95917689 +0000 UTC m=+3.078982453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs") pod "network-metrics-daemon-xh2hs" (UID: "fbb22632-1c12-4275-a177-5dbc51f21162") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:21.157669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.059899    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:21.157669 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:21.060057    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:51:21.157669 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:21.060071    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:51:21.157669 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:21.060082    2573 projected.go:196] Error preparing data for projected volume kube-api-access-z9whb for pod openshift-network-diagnostics/network-check-target-zbg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:21.157669 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:21.060139    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb podName:7cee1a78-4f58-439f-8c26-66f336634a49 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:22.060121192 +0000 UTC m=+3.179926751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z9whb" (UniqueName: "kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb") pod "network-check-target-zbg6k" (UID: "7cee1a78-4f58-439f-8c26-66f336634a49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:21.289903 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.289842    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 13:46:20 +0000 UTC" deadline="2028-04-16 08:53:42.657489276 +0000 UTC"
Jul 15 13:51:21.289903 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.289901    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15379h2m21.367590697s"
Jul 15 13:51:21.409189 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.409167    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:21.413415 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.413395    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:51:21.423043 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.423026    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c5qpr"
Jul 15 13:51:21.443153 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.443128    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qfvwg"
Jul 15 13:51:21.456224 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.456193    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vffsz"
Jul 15 13:51:21.463817 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.463664    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qt2vd"
Jul 15 13:51:21.479030 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.479012    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:21.494149 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.494120    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g"
Jul 15 13:51:21.499745 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.499726    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qjc4g"
Jul 15 13:51:21.503614 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.503599    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k5mg8"
Jul 15 13:51:21.547882 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:21.547841    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e182e8671520a5dbfd171d137a66aa6.slice/crio-08fd8c9e6fa666c9a49de4e764732061887e3b3235e0f187b4d4eeca4f3446e2 WatchSource:0}: Error finding container 08fd8c9e6fa666c9a49de4e764732061887e3b3235e0f187b4d4eeca4f3446e2: Status 404 returned error can't find the container with id 08fd8c9e6fa666c9a49de4e764732061887e3b3235e0f187b4d4eeca4f3446e2
Jul 15 13:51:21.548071 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:21.548057    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43c7d6073f1b1b5cc49a22829a91c6e.slice/crio-c17a844248d697c55197ccc710b43a4d638bce4ef3fbf36367e6dc6fddd36fb0 WatchSource:0}: Error finding container c17a844248d697c55197ccc710b43a4d638bce4ef3fbf36367e6dc6fddd36fb0: Status 404 returned error can't find the container with id c17a844248d697c55197ccc710b43a4d638bce4ef3fbf36367e6dc6fddd36fb0
Jul 15 13:51:21.551352 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.551321    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 13:51:21.770750 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:21.770722    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2520d340_bfc2_4022_ad9b_4956b7a7b028.slice/crio-d64747080cc96ec77829a46898f3c6e5c9e41fdf804e191b6fddc3151b8d1b41 WatchSource:0}: Error finding container d64747080cc96ec77829a46898f3c6e5c9e41fdf804e191b6fddc3151b8d1b41: Status 404 returned error can't find the container with id d64747080cc96ec77829a46898f3c6e5c9e41fdf804e191b6fddc3151b8d1b41
Jul 15 13:51:21.808152 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:21.808111    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e24913_c8a6_4c95_9a1c_e8755606b7ad.slice/crio-776ad677b9d29ce4d643fd0241aaa882e0ae2b16977706a37d68f47dde30482c WatchSource:0}: Error finding container 776ad677b9d29ce4d643fd0241aaa882e0ae2b16977706a37d68f47dde30482c: Status 404 returned error can't find the container with id 776ad677b9d29ce4d643fd0241aaa882e0ae2b16977706a37d68f47dde30482c
Jul 15 13:51:21.843790 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:21.843763    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d451b46_14cc_475e_9d9a_64dc6143af58.slice/crio-219ec9d9223dbd4c8a52193f5e26699ee247bb7e9b288f843e4a10cb5e89c263 WatchSource:0}: Error finding container 219ec9d9223dbd4c8a52193f5e26699ee247bb7e9b288f843e4a10cb5e89c263: Status 404 returned error can't find the container with id 219ec9d9223dbd4c8a52193f5e26699ee247bb7e9b288f843e4a10cb5e89c263
Jul 15 13:51:21.878056 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:21.878026    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118b09fc_fa2f_4f5e_9418_19ad1aabe3f1.slice/crio-54a6e46c09bc956b1182bba1498e41a65b3420ba56eda3bca040f1622338f82e WatchSource:0}: Error finding container 54a6e46c09bc956b1182bba1498e41a65b3420ba56eda3bca040f1622338f82e: Status 404 returned error can't find the container with id 54a6e46c09bc956b1182bba1498e41a65b3420ba56eda3bca040f1622338f82e
Jul 15 13:51:21.878924 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:21.878900    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba09580d_4c2a_452e_b857_c3c30f6d3593.slice/crio-ac02088ae770b2534f62c8eec3df0193651e80fadefde234f19f6ae0452854c8 WatchSource:0}: Error finding container ac02088ae770b2534f62c8eec3df0193651e80fadefde234f19f6ae0452854c8: Status 404 returned error can't find the container with id ac02088ae770b2534f62c8eec3df0193651e80fadefde234f19f6ae0452854c8
Jul 15 13:51:21.958080 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:21.958055    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8f200d_6b67_4f66_8c9a_2d9e423b9247.slice/crio-4fa3e8aa54191d3fd6cd48487ee2133d193c5f481b2c2964c0c6102bc17ff6f2 WatchSource:0}: Error finding container 4fa3e8aa54191d3fd6cd48487ee2133d193c5f481b2c2964c0c6102bc17ff6f2: Status 404 returned error can't find the container with id 4fa3e8aa54191d3fd6cd48487ee2133d193c5f481b2c2964c0c6102bc17ff6f2
Jul 15 13:51:21.968740 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.968708    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:21.968873 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:21.968747    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:21.968873 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:21.968865    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:21.968943 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:21.968924    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs podName:fbb22632-1c12-4275-a177-5dbc51f21162 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:23.968910676 +0000 UTC m=+5.088716238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs") pod "network-metrics-daemon-xh2hs" (UID: "fbb22632-1c12-4275-a177-5dbc51f21162") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:21.968982 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:21.968865    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:21.969019 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:21.968993    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret podName:6edabe15-150f-4758-8b05-a9c0f1b266bf nodeName:}" failed. No retries permitted until 2026-07-15 13:51:23.968981827 +0000 UTC m=+5.088787385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret") pod "global-pull-secret-syncer-qnhlv" (UID: "6edabe15-150f-4758-8b05-a9c0f1b266bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:22.069827 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.069736    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:22.069963 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:22.069928    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:51:22.069963 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:22.069957    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:51:22.070053 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:22.069971    2573 projected.go:196] Error preparing data for projected volume kube-api-access-z9whb for pod openshift-network-diagnostics/network-check-target-zbg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:22.070053 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:22.070036    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb podName:7cee1a78-4f58-439f-8c26-66f336634a49 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:24.070020496 +0000 UTC m=+5.189826054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z9whb" (UniqueName: "kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb") pod "network-check-target-zbg6k" (UID: "7cee1a78-4f58-439f-8c26-66f336634a49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:22.195713 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:22.195671    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40d9221_15b5_4836_83ad_42bdad956a48.slice/crio-d5ab73b302b0b3688c25455ba6fdbaa55b3eb3329ca32f71fec5773fcdf5ae9b WatchSource:0}: Error finding container d5ab73b302b0b3688c25455ba6fdbaa55b3eb3329ca32f71fec5773fcdf5ae9b: Status 404 returned error can't find the container with id d5ab73b302b0b3688c25455ba6fdbaa55b3eb3329ca32f71fec5773fcdf5ae9b
Jul 15 13:51:22.291087 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.291006    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 13:46:20 +0000 UTC" deadline="2028-02-07 17:57:27.938876334 +0000 UTC"
Jul 15 13:51:22.291087 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.291066    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13732h6m5.647814497s"
Jul 15 13:51:22.401195 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.400369    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:22.401195 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:22.400498    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:22.401195 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.400884    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:22.401195 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:22.400982    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:22.401195 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.401041    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:22.401195 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:22.401104    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:22.412723 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.412659    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pnhlx" event={"ID":"af8f200d-6b67-4f66-8c9a-2d9e423b9247","Type":"ContainerStarted","Data":"4fa3e8aa54191d3fd6cd48487ee2133d193c5f481b2c2964c0c6102bc17ff6f2"}
Jul 15 13:51:22.420471 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.420394    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vffsz" event={"ID":"b6e24913-c8a6-4c95-9a1c-e8755606b7ad","Type":"ContainerStarted","Data":"776ad677b9d29ce4d643fd0241aaa882e0ae2b16977706a37d68f47dde30482c"}
Jul 15 13:51:22.422011 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.421884    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"d64747080cc96ec77829a46898f3c6e5c9e41fdf804e191b6fddc3151b8d1b41"}
Jul 15 13:51:22.426476 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:22.426445    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d016d94_23a8_4b4f_91f8_acd413e4558e.slice/crio-77acb9682a46b2d10556012393b93eb61821c3439b97b31695d939acd75da7c4 WatchSource:0}: Error finding container 77acb9682a46b2d10556012393b93eb61821c3439b97b31695d939acd75da7c4: Status 404 returned error can't find the container with id 77acb9682a46b2d10556012393b93eb61821c3439b97b31695d939acd75da7c4
Jul 15 13:51:22.430572 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.430540    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal" event={"ID":"5e182e8671520a5dbfd171d137a66aa6","Type":"ContainerStarted","Data":"08fd8c9e6fa666c9a49de4e764732061887e3b3235e0f187b4d4eeca4f3446e2"}
Jul 15 13:51:22.435652 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.435556    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qjc4g" event={"ID":"ba09580d-4c2a-452e-b857-c3c30f6d3593","Type":"ContainerStarted","Data":"ac02088ae770b2534f62c8eec3df0193651e80fadefde234f19f6ae0452854c8"}
Jul 15 13:51:22.439695 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.439662    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k5mg8" event={"ID":"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1","Type":"ContainerStarted","Data":"54a6e46c09bc956b1182bba1498e41a65b3420ba56eda3bca040f1622338f82e"}
Jul 15 13:51:22.442772 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.442700    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qt2vd" event={"ID":"0d451b46-14cc-475e-9d9a-64dc6143af58","Type":"ContainerStarted","Data":"219ec9d9223dbd4c8a52193f5e26699ee247bb7e9b288f843e4a10cb5e89c263"}
Jul 15 13:51:22.451148 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.451044    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal" event={"ID":"e43c7d6073f1b1b5cc49a22829a91c6e","Type":"ContainerStarted","Data":"c17a844248d697c55197ccc710b43a4d638bce4ef3fbf36367e6dc6fddd36fb0"}
Jul 15 13:51:22.455182 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:22.455151    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qfvwg" event={"ID":"f40d9221-15b5-4836-83ad-42bdad956a48","Type":"ContainerStarted","Data":"d5ab73b302b0b3688c25455ba6fdbaa55b3eb3329ca32f71fec5773fcdf5ae9b"}
Jul 15 13:51:22.502612 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:51:22.502372    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794fea14_734a_4b7c_a471_a7cf6c26615b.slice/crio-dc729bda9093927f798584c69a015318879290da26282db3d433030877cd9237 WatchSource:0}: Error finding container dc729bda9093927f798584c69a015318879290da26282db3d433030877cd9237: Status 404 returned error can't find the container with id dc729bda9093927f798584c69a015318879290da26282db3d433030877cd9237
Jul 15 13:51:23.328782 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:23.328753    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:51:23.463400 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:23.463195    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g" event={"ID":"794fea14-734a-4b7c-a471-a7cf6c26615b","Type":"ContainerStarted","Data":"dc729bda9093927f798584c69a015318879290da26282db3d433030877cd9237"}
Jul 15 13:51:23.467828 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:23.467730    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c5qpr" event={"ID":"4d016d94-23a8-4b4f-91f8-acd413e4558e","Type":"ContainerStarted","Data":"77acb9682a46b2d10556012393b93eb61821c3439b97b31695d939acd75da7c4"}
Jul 15 13:51:23.984002 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:23.983918    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:23.984002 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:23.983966    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:23.984230 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:23.984098    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:23.984230 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:23.984161    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret podName:6edabe15-150f-4758-8b05-a9c0f1b266bf nodeName:}" failed. No retries permitted until 2026-07-15 13:51:27.984141655 +0000 UTC m=+9.103947217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret") pod "global-pull-secret-syncer-qnhlv" (UID: "6edabe15-150f-4758-8b05-a9c0f1b266bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:23.984230 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:23.984225    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:23.984405 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:23.984256    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs podName:fbb22632-1c12-4275-a177-5dbc51f21162 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:27.98424575 +0000 UTC m=+9.104051313 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs") pod "network-metrics-daemon-xh2hs" (UID: "fbb22632-1c12-4275-a177-5dbc51f21162") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:24.086073 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:24.085450    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:24.086073 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:24.085642    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:51:24.086073 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:24.085664    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:51:24.086073 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:24.085677    2573 projected.go:196] Error preparing data for projected volume kube-api-access-z9whb for pod openshift-network-diagnostics/network-check-target-zbg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:24.086073 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:24.085730    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb podName:7cee1a78-4f58-439f-8c26-66f336634a49 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:28.085712853 +0000 UTC m=+9.205518428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z9whb" (UniqueName: "kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb") pod "network-check-target-zbg6k" (UID: "7cee1a78-4f58-439f-8c26-66f336634a49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:24.400838 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:24.400802    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:24.401022 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:24.400860    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:24.401022 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:24.400802    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:24.401022 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:24.400929    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:24.401172 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:24.401021    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:24.401172 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:24.401107    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:24.515117 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:24.515038    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal" event={"ID":"e43c7d6073f1b1b5cc49a22829a91c6e","Type":"ContainerStarted","Data":"9ffd15f98b89559fc99afe415e57309285336ec73b79f368c2aef3dcbc98351b"}
Jul 15 13:51:25.521093 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:25.520977    2573 generic.go:356] "Generic (PLEG): container finished" podID="e43c7d6073f1b1b5cc49a22829a91c6e" containerID="9ffd15f98b89559fc99afe415e57309285336ec73b79f368c2aef3dcbc98351b" exitCode=0
Jul 15 13:51:25.521093 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:25.521026    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal" event={"ID":"e43c7d6073f1b1b5cc49a22829a91c6e","Type":"ContainerDied","Data":"9ffd15f98b89559fc99afe415e57309285336ec73b79f368c2aef3dcbc98351b"}
Jul 15 13:51:26.400924 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:26.400802    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:26.401109 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:26.400942    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:26.401109 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:26.401002    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:26.401222 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:26.401121    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:26.401222 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:26.401208    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:26.401350 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:26.401302    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:28.020726 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:28.020633    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:28.020726 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:28.020683    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:28.021239 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.020822    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:28.021239 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.020864    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:28.021239 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.020897    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret podName:6edabe15-150f-4758-8b05-a9c0f1b266bf nodeName:}" failed. No retries permitted until 2026-07-15 13:51:36.020877198 +0000 UTC m=+17.140682760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret") pod "global-pull-secret-syncer-qnhlv" (UID: "6edabe15-150f-4758-8b05-a9c0f1b266bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:28.021239 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.020929    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs podName:fbb22632-1c12-4275-a177-5dbc51f21162 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:36.020911099 +0000 UTC m=+17.140716661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs") pod "network-metrics-daemon-xh2hs" (UID: "fbb22632-1c12-4275-a177-5dbc51f21162") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:28.121685 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:28.121645    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:28.121890 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.121847    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:51:28.121984 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.121894    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:51:28.121984 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.121908    2573 projected.go:196] Error preparing data for projected volume kube-api-access-z9whb for pod openshift-network-diagnostics/network-check-target-zbg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:28.122076 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.121992    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb podName:7cee1a78-4f58-439f-8c26-66f336634a49 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:36.121962238 +0000 UTC m=+17.241767812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z9whb" (UniqueName: "kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb") pod "network-check-target-zbg6k" (UID: "7cee1a78-4f58-439f-8c26-66f336634a49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:28.401621 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:28.400948    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:28.401621 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.401071    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:28.401621 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:28.401452    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:28.401621 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.401540    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:28.401621 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:28.401589    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:28.401959 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:28.401679    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:30.400524 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:30.400486    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:30.400524 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:30.400524    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:30.401006 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:30.400628    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:30.401006 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:30.400671    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:30.401006 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:30.400716    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:30.401006 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:30.400772    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:32.400294 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:32.400205    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:32.400681 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:32.400326    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:32.400681 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:32.400369    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:32.400681 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:32.400366    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:32.400681 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:32.400448    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:32.400681 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:32.400541    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:34.401039 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:34.401005    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:34.401528 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:34.401117    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:34.401528 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:34.401133    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:34.401528 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:34.401148    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:34.401528 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:34.401384    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:34.401528 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:34.401476    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:36.084263 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:36.084208    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:36.084263 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:36.084262    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:36.084775 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.084401    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:36.084775 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.084483    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs podName:fbb22632-1c12-4275-a177-5dbc51f21162 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.084461116 +0000 UTC m=+33.204266676 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs") pod "network-metrics-daemon-xh2hs" (UID: "fbb22632-1c12-4275-a177-5dbc51f21162") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:36.084775 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.084413    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:36.084775 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.084550    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret podName:6edabe15-150f-4758-8b05-a9c0f1b266bf nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.084533992 +0000 UTC m=+33.204339554 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret") pod "global-pull-secret-syncer-qnhlv" (UID: "6edabe15-150f-4758-8b05-a9c0f1b266bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:36.185131 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:36.185091    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:36.185304 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.185271    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:51:36.185304 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.185294    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:51:36.185405 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.185309    2573 projected.go:196] Error preparing data for projected volume kube-api-access-z9whb for pod openshift-network-diagnostics/network-check-target-zbg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:36.185405 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.185397    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb podName:7cee1a78-4f58-439f-8c26-66f336634a49 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.18537758 +0000 UTC m=+33.305183138 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z9whb" (UniqueName: "kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb") pod "network-check-target-zbg6k" (UID: "7cee1a78-4f58-439f-8c26-66f336634a49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:36.400769 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:36.400687    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:36.400769 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:36.400736    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:36.400990 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:36.400691    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:36.400990 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.400804    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:36.400990 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.400915    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:36.401144 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:36.400994    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:37.404319 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:37.404243    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:37.404753 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:37.404396    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:38.400546 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:38.400510    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:38.400745 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:38.400510    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:38.400745 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:38.400640    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:38.400745 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:38.400685    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:39.401210 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:39.401175    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:39.401693 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:39.401297    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:40.401151 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:40.401113    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:40.401364 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:40.401235    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:40.401364 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:40.401267    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:40.401364 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:40.401357    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:41.401143 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:41.401126    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:41.401241 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:41.401225    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:42.400794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.400290    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:42.400794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.400370    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:42.401784 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:42.400827    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:42.401784 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:42.400964    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:42.556700 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.556511    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qjc4g" event={"ID":"ba09580d-4c2a-452e-b857-c3c30f6d3593","Type":"ContainerStarted","Data":"3265c374965a5dc67321fad6982e8a4d100b79ea6fb6dea52b1d0060a5035912"}
Jul 15 13:51:42.558285 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.558249    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k5mg8" event={"ID":"118b09fc-fa2f-4f5e-9418-19ad1aabe3f1","Type":"ContainerStarted","Data":"568ad8b1132fc4eef7226f2383bace162816fb494c8b66b9c52141c480bbb5a9"}
Jul 15 13:51:42.560126 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.560104    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal" event={"ID":"e43c7d6073f1b1b5cc49a22829a91c6e","Type":"ContainerStarted","Data":"916bd43ddce8eb69c257b723f8635033eff4d4ac6e53afaac208e82de2221ee1"}
Jul 15 13:51:42.561635 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.561615    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g" event={"ID":"794fea14-734a-4b7c-a471-a7cf6c26615b","Type":"ContainerStarted","Data":"8356330dca87423391154bb4cd2912c4f0ea1491d15a212e1db3359e37c260b2"}
Jul 15 13:51:42.563181 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.563160    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c5qpr" event={"ID":"4d016d94-23a8-4b4f-91f8-acd413e4558e","Type":"ContainerStarted","Data":"9664d0f4a18cd95a68e3e88997990e469841c403ed52fd8efed2053c7ad02a60"}
Jul 15 13:51:42.564615 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.564584    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qfvwg" event={"ID":"f40d9221-15b5-4836-83ad-42bdad956a48","Type":"ContainerStarted","Data":"24a4a64a9ef6ad1e590c0d90edaf488ea45ae17271e40929934e0f1e000edfaa"}
Jul 15 13:51:42.566074 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.566048    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pnhlx" event={"ID":"af8f200d-6b67-4f66-8c9a-2d9e423b9247","Type":"ContainerStarted","Data":"2074f9f326bcae38a714a91e072e4012880f601106ba521d49ea3e288f63f11b"}
Jul 15 13:51:42.567623 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.567601    2573 generic.go:356] "Generic (PLEG): container finished" podID="b6e24913-c8a6-4c95-9a1c-e8755606b7ad" containerID="35baa7c530cc309902f7d9fcf495c30c2b40ed09dd7e7d78ae2ec9f80ab39d9b" exitCode=0
Jul 15 13:51:42.567718 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.567685    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vffsz" event={"ID":"b6e24913-c8a6-4c95-9a1c-e8755606b7ad","Type":"ContainerDied","Data":"35baa7c530cc309902f7d9fcf495c30c2b40ed09dd7e7d78ae2ec9f80ab39d9b"}
Jul 15 13:51:42.570727 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.570709    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 13:51:42.571067 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.571046    2573 generic.go:356] "Generic (PLEG): container finished" podID="2520d340-bfc2-4022-ad9b-4956b7a7b028" containerID="c984c21c4a5743dcdbe82d27cdc1d16bbce0fe209f25af31db54197d7f152dee" exitCode=1
Jul 15 13:51:42.571156 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.571117    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"9a80e7a679a157f710d38568d9af7264d88c224f6dbba05e4f04e6987ad3127c"}
Jul 15 13:51:42.571213 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.571168    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"00606c8011ba3fed11c3eaec433b3a6869fe2b53644c54ffc86b82dcb7bf4758"}
Jul 15 13:51:42.571262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.571221    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"b227bdfd0111051355049b0a6766eb181af42dd90cae43921e28af98ad0d49ae"}
Jul 15 13:51:42.571262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.571257    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"536ffe5bbe0f752605b9753d783148aa9bfd88f0f3882a3228973d8b86db7d0a"}
Jul 15 13:51:42.571349 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.571298    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerDied","Data":"c984c21c4a5743dcdbe82d27cdc1d16bbce0fe209f25af31db54197d7f152dee"}
Jul 15 13:51:42.571399 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.571348    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"07dbc2754f53e668251d87342e5a5c12a17acb410025cf604a6126e694c07851"}
Jul 15 13:51:42.572578 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.572558    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal" event={"ID":"5e182e8671520a5dbfd171d137a66aa6","Type":"ContainerStarted","Data":"33390e90c7ead3739477da56cb11a0dd12869183724b6d41e548d51a6a1f2b24"}
Jul 15 13:51:42.581309 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.581251    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qjc4g" podStartSLOduration=3.976237913 podStartE2EDuration="23.581236528s" totalImagesPullingTime="19.604998615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:42.580767179 +0000 UTC m=+23.700572762" watchObservedRunningTime="2026-07-15 13:51:42.581236528 +0000 UTC m=+23.701042106"
Jul 15 13:51:42.600262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.600212    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c5qpr" podStartSLOduration=4.544474061 podStartE2EDuration="23.600192774s" totalImagesPullingTime="19.055718713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:42.599492138 +0000 UTC m=+23.719297717" watchObservedRunningTime="2026-07-15 13:51:42.600192774 +0000 UTC m=+23.719998354"
Jul 15 13:51:42.617253 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.617197    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qfvwg" podStartSLOduration=4.328010207 podStartE2EDuration="23.61717858s" totalImagesPullingTime="19.289168373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:42.616552992 +0000 UTC m=+23.736358573" watchObservedRunningTime="2026-07-15 13:51:42.61717858 +0000 UTC m=+23.736984160"
Jul 15 13:51:42.666079 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.665936    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pnhlx" podStartSLOduration=8.655713213 podStartE2EDuration="23.665913947s" totalImagesPullingTime="15.010200734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:42.665305207 +0000 UTC m=+23.785110824" watchObservedRunningTime="2026-07-15 13:51:42.665913947 +0000 UTC m=+23.785719530"
Jul 15 13:51:42.714350 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.714288    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k5mg8" podStartSLOduration=4.156837799 podStartE2EDuration="23.714277586s" totalImagesPullingTime="19.557439787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:42.713860349 +0000 UTC m=+23.833665930" watchObservedRunningTime="2026-07-15 13:51:42.714277586 +0000 UTC m=+23.834083164"
Jul 15 13:51:42.787038 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.786990    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-208.ec2.internal" podStartSLOduration=22.786978943 podStartE2EDuration="22.786978943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:42.785895371 +0000 UTC m=+23.905700952" watchObservedRunningTime="2026-07-15 13:51:42.786978943 +0000 UTC m=+23.906784521"
Jul 15 13:51:42.787210 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:42.787133    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-208.ec2.internal" podStartSLOduration=22.787129896 podStartE2EDuration="22.787129896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:42.749308626 +0000 UTC m=+23.869114205" watchObservedRunningTime="2026-07-15 13:51:42.787129896 +0000 UTC m=+23.906935476"
Jul 15 13:51:43.142374 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:43.142326    2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 13:51:43.331597 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:43.331214    2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T13:51:43.142365715Z","UUID":"2ebeac6b-d374-4c04-aceb-2432a5e1a39f","Handler":null,"Name":"","Endpoint":""}
Jul 15 13:51:43.336126 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:43.333527    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 13:51:43.336126 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:43.333578    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 13:51:43.401170 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:43.401127    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:43.401589 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:43.401274    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:43.576406 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:43.576366    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qt2vd" event={"ID":"0d451b46-14cc-475e-9d9a-64dc6143af58","Type":"ContainerStarted","Data":"11190df2b6abfa3d460a64d3bd05b2f8e869c409c0d2ac03a2ff2ad751b92fe8"}
Jul 15 13:51:43.578623 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:43.578523    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g" event={"ID":"794fea14-734a-4b7c-a471-a7cf6c26615b","Type":"ContainerStarted","Data":"b9e59dd66b970366fb6cc975b869fb36cd2102b4d3530629747038723d36b491"}
Jul 15 13:51:43.594040 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:43.593919    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qt2vd" podStartSLOduration=4.952913894 podStartE2EDuration="24.593902776s" totalImagesPullingTime="19.640988882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:43.593650043 +0000 UTC m=+24.713455633" watchObservedRunningTime="2026-07-15 13:51:43.593902776 +0000 UTC m=+24.713708363"
Jul 15 13:51:44.400948 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:44.400909    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:44.401159 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:44.400950    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:44.401159 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:44.401054    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:44.401538 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:44.401177    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:45.401199 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:45.401112    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:45.401828 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:45.401246    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:45.586721 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:45.586689    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 13:51:45.587155 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:45.587130    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"94f9543c60d5354fdcf26cba574a83834c3d74ed08485d9367c79b816b958f5d"}
Jul 15 13:51:45.589200 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:45.589174    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g" event={"ID":"794fea14-734a-4b7c-a471-a7cf6c26615b","Type":"ContainerStarted","Data":"6f49ca054a2c895c810872de93cdc53274def585ac3308eecbf36ccd5abd4f5d"}
Jul 15 13:51:45.610787 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:45.610724    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6m27g" podStartSLOduration=4.599494565 podStartE2EDuration="26.610703374s" totalImagesPullingTime="22.011208809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:45.609173412 +0000 UTC m=+26.728978992" watchObservedRunningTime="2026-07-15 13:51:45.610703374 +0000 UTC m=+26.730508957"
Jul 15 13:51:46.401107 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:46.401065    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:46.401401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:46.401067    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:46.401401 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:46.401186    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:46.401401 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:46.401274    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:47.401094 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.400833    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:47.401237 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:47.401212    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:47.544200 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.544111    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:47.544767 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.544697    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:47.594741 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.594712    2573 generic.go:356] "Generic (PLEG): container finished" podID="b6e24913-c8a6-4c95-9a1c-e8755606b7ad" containerID="d041da3f81c8fe9b402bbdd2629e9881bd10157935cfa069820de79e8e356882" exitCode=0
Jul 15 13:51:47.594927 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.594794    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vffsz" event={"ID":"b6e24913-c8a6-4c95-9a1c-e8755606b7ad","Type":"ContainerDied","Data":"d041da3f81c8fe9b402bbdd2629e9881bd10157935cfa069820de79e8e356882"}
Jul 15 13:51:47.598394 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.598375    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 13:51:47.598794 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.598763    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"00b59547c090f1a659e7b8be9fcfb1a7237b689bf49629c1f7bff908d556c85b"}
Jul 15 13:51:47.599071 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.599047    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:47.599151 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.599117    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:47.599151 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.599131    2573 scope.go:117] "RemoveContainer" containerID="c984c21c4a5743dcdbe82d27cdc1d16bbce0fe209f25af31db54197d7f152dee"
Jul 15 13:51:47.599464 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:47.599450    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pnhlx"
Jul 15 13:51:48.400520 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.400488    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:48.400792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.400489    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:48.400792 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:48.400617    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:48.400792 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:48.400672    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:48.605046 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.605021    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 13:51:48.605634 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.605384    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" event={"ID":"2520d340-bfc2-4022-ad9b-4956b7a7b028","Type":"ContainerStarted","Data":"0fde8125cd04c86a9c8e2b725bdc05f662b5df3cfa796612b452769064a5ce7f"}
Jul 15 13:51:48.605776 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.605756    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:48.605834 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.605826    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:48.621257 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.621230    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:48.621427 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.621366    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:51:48.646075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.646026    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d" podStartSLOduration=9.859710894 podStartE2EDuration="29.646010796s" totalImagesPullingTime="19.786299902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:48.645920615 +0000 UTC m=+29.765726195" watchObservedRunningTime="2026-07-15 13:51:48.646010796 +0000 UTC m=+29.765816377"
Jul 15 13:51:48.749978 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.749729    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k5mg8_118b09fc-fa2f-4f5e-9418-19ad1aabe3f1/dns-node-resolver/0.log"
Jul 15 13:51:48.951977 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.951899    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zbg6k"]
Jul 15 13:51:48.952128 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.951995    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:48.952128 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:48.952086    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:48.955191 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.955148    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qnhlv"]
Jul 15 13:51:48.955385 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.955270    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:48.955454 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:48.955394    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:48.955777 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.955750    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xh2hs"]
Jul 15 13:51:48.955887 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:48.955865    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:48.955985 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:48.955964    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:49.609793 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:49.609752    2573 generic.go:356] "Generic (PLEG): container finished" podID="b6e24913-c8a6-4c95-9a1c-e8755606b7ad" containerID="0846c9edc24f07e8b1370cb27bd94cbfe4aa16ae2cd56b7976b7f655949464c6" exitCode=0
Jul 15 13:51:49.610441 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:49.609850    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vffsz" event={"ID":"b6e24913-c8a6-4c95-9a1c-e8755606b7ad","Type":"ContainerDied","Data":"0846c9edc24f07e8b1370cb27bd94cbfe4aa16ae2cd56b7976b7f655949464c6"}
Jul 15 13:51:49.916727 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:49.916650    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qfvwg_f40d9221-15b5-4836-83ad-42bdad956a48/node-ca/0.log"
Jul 15 13:51:50.400744 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:50.400667    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:50.400885 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:50.400678    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:50.400885 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:50.400768    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:50.400885 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:50.400860    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:51.400973 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:51.400945    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:51.401550 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:51.401063    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:51.616233 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:51.616198    2573 generic.go:356] "Generic (PLEG): container finished" podID="b6e24913-c8a6-4c95-9a1c-e8755606b7ad" containerID="0f30e283ec12c0dd82e1fe5ca0ba4aa51504395ad9398bd5e33bfb5ea42ebef9" exitCode=0
Jul 15 13:51:51.616405 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:51.616253    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vffsz" event={"ID":"b6e24913-c8a6-4c95-9a1c-e8755606b7ad","Type":"ContainerDied","Data":"0f30e283ec12c0dd82e1fe5ca0ba4aa51504395ad9398bd5e33bfb5ea42ebef9"}
Jul 15 13:51:52.102677 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:52.102642    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:52.102677 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:52.102685    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:52.102937 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.102809    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:52.102937 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.102867    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret podName:6edabe15-150f-4758-8b05-a9c0f1b266bf nodeName:}" failed. No retries permitted until 2026-07-15 13:52:24.102850153 +0000 UTC m=+65.222655723 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret") pod "global-pull-secret-syncer-qnhlv" (UID: "6edabe15-150f-4758-8b05-a9c0f1b266bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:51:52.102937 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.102809    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:52.103083 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.102943    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs podName:fbb22632-1c12-4275-a177-5dbc51f21162 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:24.102930026 +0000 UTC m=+65.222735586 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs") pod "network-metrics-daemon-xh2hs" (UID: "fbb22632-1c12-4275-a177-5dbc51f21162") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:51:52.203342 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:52.203301    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:52.203593 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.203502    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:51:52.203593 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.203528    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:51:52.203593 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.203541    2573 projected.go:196] Error preparing data for projected volume kube-api-access-z9whb for pod openshift-network-diagnostics/network-check-target-zbg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:52.203761 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.203608    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb podName:7cee1a78-4f58-439f-8c26-66f336634a49 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:24.2035885 +0000 UTC m=+65.323394072 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z9whb" (UniqueName: "kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb") pod "network-check-target-zbg6k" (UID: "7cee1a78-4f58-439f-8c26-66f336634a49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:51:52.401178 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:52.401099    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:52.401600 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:52.401110    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:52.401600 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.401214    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:52.401600 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:52.401311    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:53.400647 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:53.400400    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:53.400827 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:53.400769    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:54.400512 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:54.400478    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:54.400921 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:54.400524    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:54.400921 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:54.400595    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:54.400921 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:54.400754    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:55.401114 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:55.401081    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:55.401600 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:55.401212    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:56.400980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:56.400945    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:56.401144 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:56.400946    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:56.401144 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:56.401050    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:56.401444 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:56.401151    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:56.632013 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:56.631980    2573 generic.go:356] "Generic (PLEG): container finished" podID="b6e24913-c8a6-4c95-9a1c-e8755606b7ad" containerID="d37b6fa1aca585db203b507308030e90554b70be6ff30bb8781f6619c622e730" exitCode=0
Jul 15 13:51:56.632180 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:56.632046    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vffsz" event={"ID":"b6e24913-c8a6-4c95-9a1c-e8755606b7ad","Type":"ContainerDied","Data":"d37b6fa1aca585db203b507308030e90554b70be6ff30bb8781f6619c622e730"}
Jul 15 13:51:57.400598 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:57.400564    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:57.400774 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:57.400681    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:51:57.637926 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:57.637893    2573 generic.go:356] "Generic (PLEG): container finished" podID="b6e24913-c8a6-4c95-9a1c-e8755606b7ad" containerID="e399237146ff9ba8bb837dbd0d2f95dbab859e2f8f30e970fc97777a02ee94af" exitCode=0
Jul 15 13:51:57.638301 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:57.637952    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vffsz" event={"ID":"b6e24913-c8a6-4c95-9a1c-e8755606b7ad","Type":"ContainerDied","Data":"e399237146ff9ba8bb837dbd0d2f95dbab859e2f8f30e970fc97777a02ee94af"}
Jul 15 13:51:58.400769 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:58.400735    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:51:58.400977 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:58.400735    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:51:58.400977 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:58.400864    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:51:58.400977 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:58.400920    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:51:58.643104 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:58.643063    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vffsz" event={"ID":"b6e24913-c8a6-4c95-9a1c-e8755606b7ad","Type":"ContainerStarted","Data":"983b1789374e6a7483cd0f430543741285b3caac092134a2053b6f9099011478"}
Jul 15 13:51:58.673200 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:58.673145    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vffsz" podStartSLOduration=9.714650188 podStartE2EDuration="39.673129451s" totalImagesPullingTime="29.958479263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:58.67264806 +0000 UTC m=+39.792453639" watchObservedRunningTime="2026-07-15 13:51:58.673129451 +0000 UTC m=+39.792935030"
Jul 15 13:51:59.401806 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:51:59.401771    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:51:59.401986 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:51:59.401859    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:52:00.401046 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:00.400999    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:00.401438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:00.400999    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:00.401438 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:00.401181    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:52:00.401438 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:00.401288    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:52:01.400820 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:01.400785    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:01.400965 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:01.400892    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:52:02.400656 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:02.400619    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:02.401101 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:02.400620    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:02.401101 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:02.400745    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:52:02.401101 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:02.400796    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:52:03.400833 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:03.400802    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:03.401179 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:03.400898    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:52:04.401083 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:04.401049    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:04.401500 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:04.401057    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:04.401500 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:04.401150    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:52:04.401500 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:04.401241    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:52:05.400571 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:05.400535    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:05.400810 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:05.400639    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:52:06.401031 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:06.400999    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:06.401423 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:06.401092    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:52:06.401423 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:06.401156    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:06.401423 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:06.401241    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:52:07.403826 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:07.403796    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:07.404179 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:07.403916    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:52:08.400296 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:08.400261    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:08.400493 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:08.400268    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:08.400493 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:08.400384    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:52:08.400493 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:08.400450    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:52:09.403219 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:09.403190    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:09.403616 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:09.403277    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:52:10.400720 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:10.400674    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:10.400720 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:10.400707    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:10.400928 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:10.400793    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zbg6k" podUID="7cee1a78-4f58-439f-8c26-66f336634a49"
Jul 15 13:52:10.400928 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:10.400919    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2hs" podUID="fbb22632-1c12-4275-a177-5dbc51f21162"
Jul 15 13:52:11.401263 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:11.401230    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:11.401730 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:11.401359    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnhlv" podUID="6edabe15-150f-4758-8b05-a9c0f1b266bf"
Jul 15 13:52:12.182675 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.182643    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-208.ec2.internal" event="NodeReady"
Jul 15 13:52:12.182828 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.182767    2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 13:52:12.243326 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.243238    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hd5jl"]
Jul 15 13:52:12.261362 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.261317    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tx4jt"]
Jul 15 13:52:12.261539 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.261448    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.266674 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.266644    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2df9f\""
Jul 15 13:52:12.266821 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.266755    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 13:52:12.267025 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.267007    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 13:52:12.280665 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.280634    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hd5jl"]
Jul 15 13:52:12.280665 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.280672    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tx4jt"]
Jul 15 13:52:12.280859 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.280766    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.283749 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.283720    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 13:52:12.283899 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.283731    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 13:52:12.283899 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.283771    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 13:52:12.283899 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.283810    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 13:52:12.283899 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.283774    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8t982\""
Jul 15 13:52:12.342033 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.341978    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g2qkr"]
Jul 15 13:52:12.357224 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.357189    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g2qkr"]
Jul 15 13:52:12.357399 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.357351    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g2qkr"
Jul 15 13:52:12.360342 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.360307    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 13:52:12.360484 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.360353    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 13:52:12.360787 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.360774    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 13:52:12.360828 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.360790    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-75k4w\""
Jul 15 13:52:12.400753 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.400717    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:12.400942 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.400717    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:12.404032 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.403892    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 13:52:12.404032 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.403919    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 13:52:12.404032 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.403924    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 13:52:12.404032 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.403979    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vzz42\""
Jul 15 13:52:12.404550 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.404251    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rdkz6\""
Jul 15 13:52:12.450977 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.450934    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-tmp-dir\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.451141 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.451006    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-config-volume\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.451141 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.451032    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-data-volume\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.451141 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.451055    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.451141 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.451080    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtvq9\" (UniqueName: \"kubernetes.io/projected/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-kube-api-access-gtvq9\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.451306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.451153    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-metrics-tls\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.451306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.451198    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.451306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.451231    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-crio-socket\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.451306 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.451246    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bfk5\" (UniqueName: \"kubernetes.io/projected/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-kube-api-access-8bfk5\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.552481 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552445    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.552481 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552485    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a318d24-fe02-4cc0-a435-f54379e199cb-cert\") pod \"ingress-canary-g2qkr\" (UID: \"1a318d24-fe02-4cc0-a435-f54379e199cb\") " pod="openshift-ingress-canary/ingress-canary-g2qkr"
Jul 15 13:52:12.552716 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552509    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-crio-socket\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.552716 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552524    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bfk5\" (UniqueName: \"kubernetes.io/projected/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-kube-api-access-8bfk5\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.552716 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552554    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-tmp-dir\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.552716 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552609    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-config-volume\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.552716 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552628    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-data-volume\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.552716 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552649    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.552716 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552673    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mmzc\" (UniqueName: \"kubernetes.io/projected/1a318d24-fe02-4cc0-a435-f54379e199cb-kube-api-access-2mmzc\") pod \"ingress-canary-g2qkr\" (UID: \"1a318d24-fe02-4cc0-a435-f54379e199cb\") " pod="openshift-ingress-canary/ingress-canary-g2qkr"
Jul 15 13:52:12.552716 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552702    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtvq9\" (UniqueName: \"kubernetes.io/projected/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-kube-api-access-gtvq9\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.553052 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552740    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-metrics-tls\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.553052 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552738    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-crio-socket\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.553052 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.552929    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-tmp-dir\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.553206 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.553109    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-data-volume\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.553268 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.553206    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.553268 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.553209    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-config-volume\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.556533 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.556505    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.556666 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.556591    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-metrics-tls\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.561439 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.561411    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtvq9\" (UniqueName: \"kubernetes.io/projected/ba4d24b8-6680-4674-a0a1-4cc0dc49948a-kube-api-access-gtvq9\") pod \"dns-default-hd5jl\" (UID: \"ba4d24b8-6680-4674-a0a1-4cc0dc49948a\") " pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.561586 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.561568    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bfk5\" (UniqueName: \"kubernetes.io/projected/3e8fe6a3-b050-46e6-b2b6-107ab3300d83-kube-api-access-8bfk5\") pod \"insights-runtime-extractor-tx4jt\" (UID: \"3e8fe6a3-b050-46e6-b2b6-107ab3300d83\") " pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.571521 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.571491    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:12.589659 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.589629    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tx4jt"
Jul 15 13:52:12.654010 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.653954    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mmzc\" (UniqueName: \"kubernetes.io/projected/1a318d24-fe02-4cc0-a435-f54379e199cb-kube-api-access-2mmzc\") pod \"ingress-canary-g2qkr\" (UID: \"1a318d24-fe02-4cc0-a435-f54379e199cb\") " pod="openshift-ingress-canary/ingress-canary-g2qkr"
Jul 15 13:52:12.654202 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.654053    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a318d24-fe02-4cc0-a435-f54379e199cb-cert\") pod \"ingress-canary-g2qkr\" (UID: \"1a318d24-fe02-4cc0-a435-f54379e199cb\") " pod="openshift-ingress-canary/ingress-canary-g2qkr"
Jul 15 13:52:12.657406 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.657377    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a318d24-fe02-4cc0-a435-f54379e199cb-cert\") pod \"ingress-canary-g2qkr\" (UID: \"1a318d24-fe02-4cc0-a435-f54379e199cb\") " pod="openshift-ingress-canary/ingress-canary-g2qkr"
Jul 15 13:52:12.662643 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.662618    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mmzc\" (UniqueName: \"kubernetes.io/projected/1a318d24-fe02-4cc0-a435-f54379e199cb-kube-api-access-2mmzc\") pod \"ingress-canary-g2qkr\" (UID: \"1a318d24-fe02-4cc0-a435-f54379e199cb\") " pod="openshift-ingress-canary/ingress-canary-g2qkr"
Jul 15 13:52:12.666427 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.666402    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g2qkr"
Jul 15 13:52:12.796478 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.796291    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tx4jt"]
Jul 15 13:52:12.806682 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.806653    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hd5jl"]
Jul 15 13:52:12.811040 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:12.811010    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4d24b8_6680_4674_a0a1_4cc0dc49948a.slice/crio-437c71f8cfd01b53b62cd42a536947d514af2df6862528cc6ed1569110ef0e4c WatchSource:0}: Error finding container 437c71f8cfd01b53b62cd42a536947d514af2df6862528cc6ed1569110ef0e4c: Status 404 returned error can't find the container with id 437c71f8cfd01b53b62cd42a536947d514af2df6862528cc6ed1569110ef0e4c
Jul 15 13:52:12.833717 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:12.833685    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g2qkr"]
Jul 15 13:52:12.838005 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:12.837968    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a318d24_fe02_4cc0_a435_f54379e199cb.slice/crio-56ab879fdb5acaef5b9af5455793a1945f67b94c39f308733c896b19dc7e7e4f WatchSource:0}: Error finding container 56ab879fdb5acaef5b9af5455793a1945f67b94c39f308733c896b19dc7e7e4f: Status 404 returned error can't find the container with id 56ab879fdb5acaef5b9af5455793a1945f67b94c39f308733c896b19dc7e7e4f
Jul 15 13:52:13.404900 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:13.404868    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:13.407975 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:13.407888    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 13:52:13.681940 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:13.681851    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tx4jt" event={"ID":"3e8fe6a3-b050-46e6-b2b6-107ab3300d83","Type":"ContainerStarted","Data":"ff1c65ab2a383faa04fc33b85105e09e07702014f8138f4676d20165179bf199"}
Jul 15 13:52:13.683764 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:13.683721    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hd5jl" event={"ID":"ba4d24b8-6680-4674-a0a1-4cc0dc49948a","Type":"ContainerStarted","Data":"437c71f8cfd01b53b62cd42a536947d514af2df6862528cc6ed1569110ef0e4c"}
Jul 15 13:52:13.686080 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:13.685991    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g2qkr" event={"ID":"1a318d24-fe02-4cc0-a435-f54379e199cb","Type":"ContainerStarted","Data":"56ab879fdb5acaef5b9af5455793a1945f67b94c39f308733c896b19dc7e7e4f"}
Jul 15 13:52:17.355814 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:17.355778    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"]
Jul 15 13:52:17.519849 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:17.519806    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"
Jul 15 13:52:17.522897 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:17.522861    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 15 13:52:17.523038 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:17.522911    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mfjb8\""
Jul 15 13:52:17.524311 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:17.524291    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"]
Jul 15 13:52:17.587706 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:17.587665    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/743d3c92-54e4-44fd-8242-d1a75ef6b1d5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fbz6l\" (UID: \"743d3c92-54e4-44fd-8242-d1a75ef6b1d5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"
Jul 15 13:52:17.688672 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:17.688581    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/743d3c92-54e4-44fd-8242-d1a75ef6b1d5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fbz6l\" (UID: \"743d3c92-54e4-44fd-8242-d1a75ef6b1d5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"
Jul 15 13:52:17.688829 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:17.688753    2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 15 13:52:17.688888 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:17.688852    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/743d3c92-54e4-44fd-8242-d1a75ef6b1d5-tls-certificates podName:743d3c92-54e4-44fd-8242-d1a75ef6b1d5 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:18.188829426 +0000 UTC m=+59.308634997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/743d3c92-54e4-44fd-8242-d1a75ef6b1d5-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-fbz6l" (UID: "743d3c92-54e4-44fd-8242-d1a75ef6b1d5") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 15 13:52:18.193579 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:18.193545    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/743d3c92-54e4-44fd-8242-d1a75ef6b1d5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fbz6l\" (UID: \"743d3c92-54e4-44fd-8242-d1a75ef6b1d5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"
Jul 15 13:52:18.195886 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:18.195860    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/743d3c92-54e4-44fd-8242-d1a75ef6b1d5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fbz6l\" (UID: \"743d3c92-54e4-44fd-8242-d1a75ef6b1d5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"
Jul 15 13:52:18.431285 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:18.431166    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"
Jul 15 13:52:18.938637 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:18.938549    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"]
Jul 15 13:52:18.956947 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:18.956915    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod743d3c92_54e4_44fd_8242_d1a75ef6b1d5.slice/crio-12ee63c72e44861db333976017a65895f4acfc4799d117abace4b5b18e3214a0 WatchSource:0}: Error finding container 12ee63c72e44861db333976017a65895f4acfc4799d117abace4b5b18e3214a0: Status 404 returned error can't find the container with id 12ee63c72e44861db333976017a65895f4acfc4799d117abace4b5b18e3214a0
Jul 15 13:52:19.708324 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:19.707661    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g2qkr" event={"ID":"1a318d24-fe02-4cc0-a435-f54379e199cb","Type":"ContainerStarted","Data":"d6d98bd5c7bfe9bcdb283889cebff6015501445b3500105792f2806705bf51f5"}
Jul 15 13:52:19.711688 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:19.711609    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tx4jt" event={"ID":"3e8fe6a3-b050-46e6-b2b6-107ab3300d83","Type":"ContainerStarted","Data":"50f492c1b1f5420ce3f6f11020f4b5153d8d3f69f22f45ba2cbc97da84cd9384"}
Jul 15 13:52:19.714864 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:19.714835    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hd5jl" event={"ID":"ba4d24b8-6680-4674-a0a1-4cc0dc49948a","Type":"ContainerStarted","Data":"0248b2cc8a56ad12c9e4c0048d9887747fb2fea1549718a0294fb496665279d0"}
Jul 15 13:52:19.714985 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:19.714891    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hd5jl" event={"ID":"ba4d24b8-6680-4674-a0a1-4cc0dc49948a","Type":"ContainerStarted","Data":"0c5e4cac1105b15ca60d07719fdf120c157abf6b57e4a5772dcc2b3427d0bc95"}
Jul 15 13:52:19.715115 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:19.715097    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:19.717398 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:19.717318    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l" event={"ID":"743d3c92-54e4-44fd-8242-d1a75ef6b1d5","Type":"ContainerStarted","Data":"12ee63c72e44861db333976017a65895f4acfc4799d117abace4b5b18e3214a0"}
Jul 15 13:52:19.726953 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:19.726800    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g2qkr" podStartSLOduration=1.790071666 podStartE2EDuration="7.7267816s" totalImagesPullingTime="5.936709934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:19.725399006 +0000 UTC m=+60.845204586" watchObservedRunningTime="2026-07-15 13:52:19.7267816 +0000 UTC m=+60.846587181"
Jul 15 13:52:19.745945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:19.745886    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hd5jl" podStartSLOduration=1.781851113 podStartE2EDuration="7.745863141s" totalImagesPullingTime="5.964012028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:19.745317753 +0000 UTC m=+60.865123342" watchObservedRunningTime="2026-07-15 13:52:19.745863141 +0000 UTC m=+60.865668723"
Jul 15 13:52:20.624026 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:20.623935    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mwc8d"
Jul 15 13:52:20.722222 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:20.722186    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l" event={"ID":"743d3c92-54e4-44fd-8242-d1a75ef6b1d5","Type":"ContainerStarted","Data":"66687d8a01203ca3f51e6f9c1849b3166880ab7177d5e9ad20f403d9d760f397"}
Jul 15 13:52:20.722680 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:20.722572    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"
Jul 15 13:52:20.728824 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:20.728798    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l"
Jul 15 13:52:20.745367 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:20.745299    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fbz6l" podStartSLOduration=2.431243863 podStartE2EDuration="3.745281668s" totalImagesPullingTime="1.314037805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:20.74456737 +0000 UTC m=+61.864372952" watchObservedRunningTime="2026-07-15 13:52:20.745281668 +0000 UTC m=+61.865087246"
Jul 15 13:52:21.429490 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.429456    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lflmd"]
Jul 15 13:52:21.440864 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.440831    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.444346 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.444285    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 15 13:52:21.444346 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.444306    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 13:52:21.445481 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.445458    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 13:52:21.445668 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.445646    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 15 13:52:21.445769 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.445702    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 13:52:21.445769 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.445762    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-f82pm\""
Jul 15 13:52:21.450518 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.450490    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lflmd"]
Jul 15 13:52:21.519811 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.519776    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.519811 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.519814    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.520071 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.519840    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcxrr\" (UniqueName: \"kubernetes.io/projected/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-kube-api-access-pcxrr\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.520071 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.519977    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.620980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.620935    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.621176 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.620996    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.621176 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.621026    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcxrr\" (UniqueName: \"kubernetes.io/projected/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-kube-api-access-pcxrr\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.621176 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.621145    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.621776 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.621735    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.623755 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.623728    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.623867 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.623835    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.630449 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.630422    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcxrr\" (UniqueName: \"kubernetes.io/projected/3d2dfcf1-ded7-4f19-a0db-fbe26a91e480-kube-api-access-pcxrr\") pod \"prometheus-operator-64785b6675-lflmd\" (UID: \"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:21.752378 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:21.752278    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd"
Jul 15 13:52:22.536032 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:22.535108    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lflmd"]
Jul 15 13:52:22.539569 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:22.539540    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2dfcf1_ded7_4f19_a0db_fbe26a91e480.slice/crio-40dd78e38f1c643889f7353a013a74f6957b2bc69c8b1b4bb586f2a598110527 WatchSource:0}: Error finding container 40dd78e38f1c643889f7353a013a74f6957b2bc69c8b1b4bb586f2a598110527: Status 404 returned error can't find the container with id 40dd78e38f1c643889f7353a013a74f6957b2bc69c8b1b4bb586f2a598110527
Jul 15 13:52:22.729812 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:22.729729    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tx4jt" event={"ID":"3e8fe6a3-b050-46e6-b2b6-107ab3300d83","Type":"ContainerStarted","Data":"23d07c9daae776affe4e5c4524c0a0e7b023a25d6cd0469592a06413b2026391"}
Jul 15 13:52:22.730681 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:22.730656    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd" event={"ID":"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480","Type":"ContainerStarted","Data":"40dd78e38f1c643889f7353a013a74f6957b2bc69c8b1b4bb586f2a598110527"}
Jul 15 13:52:23.602894 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.602853    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-dx9lz"]
Jul 15 13:52:23.605079 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.605053    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-dx9lz"
Jul 15 13:52:23.610087 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.609893    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 13:52:23.610221 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.610166    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-p7bqz\""
Jul 15 13:52:23.610299 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.610246    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 13:52:23.617892 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.617863    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-dx9lz"]
Jul 15 13:52:23.635247 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.635211    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw6dk\" (UniqueName: \"kubernetes.io/projected/e7fe77c2-bfaf-4163-83b8-24a2cba8b380-kube-api-access-xw6dk\") pod \"downloads-68c9cbb497-dx9lz\" (UID: \"e7fe77c2-bfaf-4163-83b8-24a2cba8b380\") " pod="openshift-console/downloads-68c9cbb497-dx9lz"
Jul 15 13:52:23.736602 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.736563    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw6dk\" (UniqueName: \"kubernetes.io/projected/e7fe77c2-bfaf-4163-83b8-24a2cba8b380-kube-api-access-xw6dk\") pod \"downloads-68c9cbb497-dx9lz\" (UID: \"e7fe77c2-bfaf-4163-83b8-24a2cba8b380\") " pod="openshift-console/downloads-68c9cbb497-dx9lz"
Jul 15 13:52:23.746329 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.746297    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw6dk\" (UniqueName: \"kubernetes.io/projected/e7fe77c2-bfaf-4163-83b8-24a2cba8b380-kube-api-access-xw6dk\") pod \"downloads-68c9cbb497-dx9lz\" (UID: \"e7fe77c2-bfaf-4163-83b8-24a2cba8b380\") " pod="openshift-console/downloads-68c9cbb497-dx9lz"
Jul 15 13:52:23.916233 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:23.916206    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-dx9lz"
Jul 15 13:52:24.067458 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.067408    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-dx9lz"]
Jul 15 13:52:24.070762 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:24.070733    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fe77c2_bfaf_4163_83b8_24a2cba8b380.slice/crio-6685729dc4ef4bb2e144df8367e4cb7703cc8661e1f54531d940805590015d6e WatchSource:0}: Error finding container 6685729dc4ef4bb2e144df8367e4cb7703cc8661e1f54531d940805590015d6e: Status 404 returned error can't find the container with id 6685729dc4ef4bb2e144df8367e4cb7703cc8661e1f54531d940805590015d6e
Jul 15 13:52:24.140048 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.139970    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:24.140048 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.140010    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:24.144144 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.143925    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 13:52:24.145719 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.145700    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 13:52:24.153240 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.153214    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb22632-1c12-4275-a177-5dbc51f21162-metrics-certs\") pod \"network-metrics-daemon-xh2hs\" (UID: \"fbb22632-1c12-4275-a177-5dbc51f21162\") " pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:24.153383 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.153239    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6edabe15-150f-4758-8b05-a9c0f1b266bf-original-pull-secret\") pod \"global-pull-secret-syncer-qnhlv\" (UID: \"6edabe15-150f-4758-8b05-a9c0f1b266bf\") " pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:24.217551 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.217518    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnhlv"
Jul 15 13:52:24.240395 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.240362    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:24.243615 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.243588    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 13:52:24.253721 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.253689    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 13:52:24.264032 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.263989    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9whb\" (UniqueName: \"kubernetes.io/projected/7cee1a78-4f58-439f-8c26-66f336634a49-kube-api-access-z9whb\") pod \"network-check-target-zbg6k\" (UID: \"7cee1a78-4f58-439f-8c26-66f336634a49\") " pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:24.360009 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.359969    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qnhlv"]
Jul 15 13:52:24.362321 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:24.362297    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6edabe15_150f_4758_8b05_a9c0f1b266bf.slice/crio-8dcad238e78435ce56b6dbe8d6bc2257738a44a0733c26a1a5da6375f9316096 WatchSource:0}: Error finding container 8dcad238e78435ce56b6dbe8d6bc2257738a44a0733c26a1a5da6375f9316096: Status 404 returned error can't find the container with id 8dcad238e78435ce56b6dbe8d6bc2257738a44a0733c26a1a5da6375f9316096
Jul 15 13:52:24.414410 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.414306    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vzz42\""
Jul 15 13:52:24.418248 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.418223    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rdkz6\""
Jul 15 13:52:24.421947 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.421926    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:24.425532 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.425513    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2hs"
Jul 15 13:52:24.587888 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.587853    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zbg6k"]
Jul 15 13:52:24.609165 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.609136    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xh2hs"]
Jul 15 13:52:24.739809 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.739717    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-dx9lz" event={"ID":"e7fe77c2-bfaf-4163-83b8-24a2cba8b380","Type":"ContainerStarted","Data":"6685729dc4ef4bb2e144df8367e4cb7703cc8661e1f54531d940805590015d6e"}
Jul 15 13:52:24.741705 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.741664    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd" event={"ID":"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480","Type":"ContainerStarted","Data":"d4e9aeaf2d147e3680703ef44b5b8db757114d703e4552ca41f58db873cce11c"}
Jul 15 13:52:24.741883 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.741727    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd" event={"ID":"3d2dfcf1-ded7-4f19-a0db-fbe26a91e480","Type":"ContainerStarted","Data":"f6b768d1586d54bcb755f439163f9b2f0414b8ff204bf83b041d187cca34f895"}
Jul 15 13:52:24.743070 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.743044    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qnhlv" event={"ID":"6edabe15-150f-4758-8b05-a9c0f1b266bf","Type":"ContainerStarted","Data":"8dcad238e78435ce56b6dbe8d6bc2257738a44a0733c26a1a5da6375f9316096"}
Jul 15 13:52:24.769620 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:24.769554    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-lflmd" podStartSLOduration=2.429394729 podStartE2EDuration="3.769534058s" totalImagesPullingTime="1.340139329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:24.767551647 +0000 UTC m=+65.887357235" watchObservedRunningTime="2026-07-15 13:52:24.769534058 +0000 UTC m=+65.889339643"
Jul 15 13:52:26.099737 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:26.099703    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb22632_1c12_4275_a177_5dbc51f21162.slice/crio-9e990f903cebdf11508f7f5ff241899de840d284c0f09669b58986279aebb129 WatchSource:0}: Error finding container 9e990f903cebdf11508f7f5ff241899de840d284c0f09669b58986279aebb129: Status 404 returned error can't find the container with id 9e990f903cebdf11508f7f5ff241899de840d284c0f09669b58986279aebb129
Jul 15 13:52:26.100172 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:26.100156    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cee1a78_4f58_439f_8c26_66f336634a49.slice/crio-d256915558654cbb45ebaa32adc2d5c6ee12382fd506f4cbecfa203b56071dfe WatchSource:0}: Error finding container d256915558654cbb45ebaa32adc2d5c6ee12382fd506f4cbecfa203b56071dfe: Status 404 returned error can't find the container with id d256915558654cbb45ebaa32adc2d5c6ee12382fd506f4cbecfa203b56071dfe
Jul 15 13:52:26.754572 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.754525    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tx4jt" event={"ID":"3e8fe6a3-b050-46e6-b2b6-107ab3300d83","Type":"ContainerStarted","Data":"1649be446ce13d89394f046e2714f09f2e6ea383c6f6fc519037bb58080ce324"}
Jul 15 13:52:26.756949 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.756912    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xh2hs" event={"ID":"fbb22632-1c12-4275-a177-5dbc51f21162","Type":"ContainerStarted","Data":"9e990f903cebdf11508f7f5ff241899de840d284c0f09669b58986279aebb129"}
Jul 15 13:52:26.759099 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.759067    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zbg6k" event={"ID":"7cee1a78-4f58-439f-8c26-66f336634a49","Type":"ContainerStarted","Data":"d256915558654cbb45ebaa32adc2d5c6ee12382fd506f4cbecfa203b56071dfe"}
Jul 15 13:52:26.838598 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.838027    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tx4jt" podStartSLOduration=1.651155475 podStartE2EDuration="14.837992174s" totalImagesPullingTime="13.186836699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:52:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:26.836698282 +0000 UTC m=+67.956503865" watchObservedRunningTime="2026-07-15 13:52:26.837992174 +0000 UTC m=+67.957797756"
Jul 15 13:52:26.889972 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.888862    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"]
Jul 15 13:52:26.891604 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.891531    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:26.892074 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.891422    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"]
Jul 15 13:52:26.895398 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.895140    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 15 13:52:26.896068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.895654    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 15 13:52:26.896068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.895877    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-df954\""
Jul 15 13:52:26.896968 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.896943    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:26.903210 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.901894    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 15 13:52:26.903210 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.902684    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 15 13:52:26.903210 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.902771    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 15 13:52:26.903210 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.902924    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7dcx4\""
Jul 15 13:52:26.905280 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.904123    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"]
Jul 15 13:52:26.923296 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.923254    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"]
Jul 15 13:52:26.923529 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.923512    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lh55b"]
Jul 15 13:52:26.927300 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.927279    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.935380 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.931767    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 13:52:26.935380 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.932013    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-64bzp\""
Jul 15 13:52:26.935380 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.932183    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 13:52:26.935380 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.932472    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 13:52:26.961027 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.960980    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15e901bb-7e01-4872-b122-7782c92e4177-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:26.961196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961036    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84d5d1ec-0377-44a4-a602-a27b899f346b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:26.961196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961067    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:26.961196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961093    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-tls\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961122    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961152    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:26.961196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961181    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-accelerators-collector-config\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961407 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961214    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlwhh\" (UniqueName: \"kubernetes.io/projected/15e901bb-7e01-4872-b122-7782c92e4177-kube-api-access-vlwhh\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:26.961407 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961239    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlsnn\" (UniqueName: \"kubernetes.io/projected/4c6a1483-128b-4350-bf4e-11782be7ae88-kube-api-access-nlsnn\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961407 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961267    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/84d5d1ec-0377-44a4-a602-a27b899f346b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:26.961407 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961292    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-root\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961407 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961350    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-wtmp\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961407 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961381    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-textfile\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961407 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961401    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/15e901bb-7e01-4872-b122-7782c92e4177-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:26.961598 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961417    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:26.961598 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961439    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-sys\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961598 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961459    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c6a1483-128b-4350-bf4e-11782be7ae88-metrics-client-ca\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:26.961598 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961489    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k45w6\" (UniqueName: \"kubernetes.io/projected/84d5d1ec-0377-44a4-a602-a27b899f346b-kube-api-access-k45w6\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:26.961598 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:26.961514    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/84d5d1ec-0377-44a4-a602-a27b899f346b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.062541 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062499    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15e901bb-7e01-4872-b122-7782c92e4177-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.062732 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062552    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84d5d1ec-0377-44a4-a602-a27b899f346b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.062732 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062581    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.062732 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062608    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-tls\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.062732 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062637    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.062732 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062669    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.062732 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062703    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-accelerators-collector-config\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062734    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwhh\" (UniqueName: \"kubernetes.io/projected/15e901bb-7e01-4872-b122-7782c92e4177-kube-api-access-vlwhh\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062783    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlsnn\" (UniqueName: \"kubernetes.io/projected/4c6a1483-128b-4350-bf4e-11782be7ae88-kube-api-access-nlsnn\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062808    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/84d5d1ec-0377-44a4-a602-a27b899f346b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062830    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-root\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062871    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-wtmp\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062900    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-textfile\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062936    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/15e901bb-7e01-4872-b122-7782c92e4177-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.062974    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.063004    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-sys\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.063036 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.063035    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c6a1483-128b-4350-bf4e-11782be7ae88-metrics-client-ca\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.063517 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.063070    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k45w6\" (UniqueName: \"kubernetes.io/projected/84d5d1ec-0377-44a4-a602-a27b899f346b-kube-api-access-k45w6\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.063517 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.063103    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/84d5d1ec-0377-44a4-a602-a27b899f346b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.063517 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.063226    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-textfile\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.063517 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:27.063305    2573 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:27.064043    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-tls podName:15e901bb-7e01-4872-b122-7782c92e4177 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:27.564016917 +0000 UTC m=+68.683822494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-x8xjp" (UID: "15e901bb-7e01-4872-b122-7782c92e4177") : secret "kube-state-metrics-tls" not found
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.064428    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84d5d1ec-0377-44a4-a602-a27b899f346b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.064807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-wtmp\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.064810    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.064844    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-root\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.064846    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-accelerators-collector-config\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:27.064909    2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:27.064954    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-tls podName:4c6a1483-128b-4350-bf4e-11782be7ae88 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:27.564940265 +0000 UTC m=+68.684745837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-tls") pod "node-exporter-lh55b" (UID: "4c6a1483-128b-4350-bf4e-11782be7ae88") : secret "node-exporter-tls" not found
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.064966    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15e901bb-7e01-4872-b122-7782c92e4177-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.065150 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.064997    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c6a1483-128b-4350-bf4e-11782be7ae88-sys\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.065756 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.065258    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/15e901bb-7e01-4872-b122-7782c92e4177-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.065756 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.065724    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c6a1483-128b-4350-bf4e-11782be7ae88-metrics-client-ca\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.067799 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.067747    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/84d5d1ec-0377-44a4-a602-a27b899f346b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.068447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.068404    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/84d5d1ec-0377-44a4-a602-a27b899f346b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.069973 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.069927    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.070088 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.069992    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.088511 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.087693    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlwhh\" (UniqueName: \"kubernetes.io/projected/15e901bb-7e01-4872-b122-7782c92e4177-kube-api-access-vlwhh\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.089303 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.089275    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k45w6\" (UniqueName: \"kubernetes.io/projected/84d5d1ec-0377-44a4-a602-a27b899f346b-kube-api-access-k45w6\") pod \"openshift-state-metrics-84bd68b896-7ss4m\" (UID: \"84d5d1ec-0377-44a4-a602-a27b899f346b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.090221 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.090194    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlsnn\" (UniqueName: \"kubernetes.io/projected/4c6a1483-128b-4350-bf4e-11782be7ae88-kube-api-access-nlsnn\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.209112 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.209076    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"
Jul 15 13:52:27.413269 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.413177    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m"]
Jul 15 13:52:27.568043 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.567993    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-tls\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.568221 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.568051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.571771 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.571693    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/15e901bb-7e01-4872-b122-7782c92e4177-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-x8xjp\" (UID: \"15e901bb-7e01-4872-b122-7782c92e4177\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.573298 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.573247    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c6a1483-128b-4350-bf4e-11782be7ae88-node-exporter-tls\") pod \"node-exporter-lh55b\" (UID: \"4c6a1483-128b-4350-bf4e-11782be7ae88\") " pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.767556 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.766808    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xh2hs" event={"ID":"fbb22632-1c12-4275-a177-5dbc51f21162","Type":"ContainerStarted","Data":"58b8d7edb243c5333affc2d09011a946fb9f271fcb2c1277580271601cbc2c98"}
Jul 15 13:52:27.767556 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.766865    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xh2hs" event={"ID":"fbb22632-1c12-4275-a177-5dbc51f21162","Type":"ContainerStarted","Data":"2da66a268b3bc1144aeff8bd9be6beda511daa9d425fbad8c570ae67b9ef9202"}
Jul 15 13:52:27.771238 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.770687    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m" event={"ID":"84d5d1ec-0377-44a4-a602-a27b899f346b","Type":"ContainerStarted","Data":"7e67b1a602ea1037c28c265e7fc1c700523c9562c9566159ca1e7a4019f5ed2c"}
Jul 15 13:52:27.771238 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.770761    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m" event={"ID":"84d5d1ec-0377-44a4-a602-a27b899f346b","Type":"ContainerStarted","Data":"893b38061cbd1bb874ff94555d491f83ac2a0947600ddd761f6ea8bc62ff7316"}
Jul 15 13:52:27.771238 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.770790    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m" event={"ID":"84d5d1ec-0377-44a4-a602-a27b899f346b","Type":"ContainerStarted","Data":"7c279a3e6d05bc627677d1e3b7bdc230289c7b250cebd020939cb593b816d89f"}
Jul 15 13:52:27.790844 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.790716    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xh2hs" podStartSLOduration=67.632087737 podStartE2EDuration="1m8.790689035s" totalImagesPullingTime="1.158601298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:27.788683355 +0000 UTC m=+68.908488929" watchObservedRunningTime="2026-07-15 13:52:27.790689035 +0000 UTC m=+68.910494615"
Jul 15 13:52:27.822236 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.822194    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"
Jul 15 13:52:27.845469 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.845372    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lh55b"
Jul 15 13:52:27.859911 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:27.859871    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c6a1483_128b_4350_bf4e_11782be7ae88.slice/crio-12edbcd31661651648406835154883f49a4c6443e4b82172a017cc17e9af1afa WatchSource:0}: Error finding container 12edbcd31661651648406835154883f49a4c6443e4b82172a017cc17e9af1afa: Status 404 returned error can't find the container with id 12edbcd31661651648406835154883f49a4c6443e4b82172a017cc17e9af1afa
Jul 15 13:52:27.912872 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.912833    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:52:27.920088 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.920057    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.927193 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.927166    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 15 13:52:27.927815 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.927792    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pb8lv\""
Jul 15 13:52:27.928010 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.927993    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 15 13:52:27.928562 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.928362    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 15 13:52:27.928838 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.928706    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 15 13:52:27.930543 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.930240    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 15 13:52:27.930543 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.930386    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 15 13:52:27.930698 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.930591    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 15 13:52:27.930855 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.930724    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 15 13:52:27.931541 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.931030    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 15 13:52:27.945982 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.945868    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:52:27.972185 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.972142    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4gs2\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-kube-api-access-p4gs2\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.972362 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.972206    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.972362 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.972234    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.972362 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.972273    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-config-out\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.972362 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.972303    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-web-config\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.972525 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.972368    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-config-volume\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.972971 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.972942    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.973085 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.972998    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.973085 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.973076    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.973196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.973106    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.973196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.973141    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.973196 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.973165    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:27.973361 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:27.973203    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.004695 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.004606    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-x8xjp"]
Jul 15 13:52:28.008327 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:28.008299    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e901bb_7e01_4872_b122_7782c92e4177.slice/crio-acff0047051ad788b4bd8f86d0fbc87074f9b00102ef3b27eaaa8f5431c636d7 WatchSource:0}: Error finding container acff0047051ad788b4bd8f86d0fbc87074f9b00102ef3b27eaaa8f5431c636d7: Status 404 returned error can't find the container with id acff0047051ad788b4bd8f86d0fbc87074f9b00102ef3b27eaaa8f5431c636d7
Jul 15 13:52:28.074574 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074536    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.074574 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074582    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.074826 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074608    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.074826 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074642    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4gs2\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-kube-api-access-p4gs2\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.074826 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074670    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.074826 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074696    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.074826 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074750    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-config-out\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.074826 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074798    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-web-config\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.074826 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074827    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-config-volume\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.075139 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074853    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.075139 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074893    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.075139 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074927    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.075139 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.074956    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.076983 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.075702    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.076983 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:28.076137    2573 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 15 13:52:28.076983 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:52:28.076207    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls podName:68c04fab-220d-499e-9345-9d4cc8f79198 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:28.576184302 +0000 UTC m=+69.695989873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198") : secret "alertmanager-main-tls" not found
Jul 15 13:52:28.077348 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.077303    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.077781 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.077412    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.078020 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.077983    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.078233 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.078209    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-config-volume\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.079780 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.079758    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-config-out\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.079982 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.079961    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-web-config\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.080880 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.080831    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.081680 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.081379    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.081680 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.081643    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.082034 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.082015    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.085949 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.085927    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4gs2\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-kube-api-access-p4gs2\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.581175 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.580626    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.584346 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.584280    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.777654 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.777610    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp" event={"ID":"15e901bb-7e01-4872-b122-7782c92e4177","Type":"ContainerStarted","Data":"acff0047051ad788b4bd8f86d0fbc87074f9b00102ef3b27eaaa8f5431c636d7"}
Jul 15 13:52:28.779327 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.779292    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lh55b" event={"ID":"4c6a1483-128b-4350-bf4e-11782be7ae88","Type":"ContainerStarted","Data":"12edbcd31661651648406835154883f49a4c6443e4b82172a017cc17e9af1afa"}
Jul 15 13:52:28.839564 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.839455    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:52:28.913746 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.913715    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6969448c8b-z8cjs"]
Jul 15 13:52:28.934918 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.934867    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6969448c8b-z8cjs"]
Jul 15 13:52:28.935109 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.935000    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:28.938848 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.938817    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 15 13:52:28.938848 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.938847    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 15 13:52:28.939074 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.938897    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 15 13:52:28.939074 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.938847    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 15 13:52:28.939648 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.939348    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zkrpd\""
Jul 15 13:52:28.939648 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.939365    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 15 13:52:28.939648 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.939352    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2mjerc57pdmru\""
Jul 15 13:52:28.984754 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.984672    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-grpc-tls\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:28.984754 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.984724    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:28.985010 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.984769    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:28.985010 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.984803    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jtgh\" (UniqueName: \"kubernetes.io/projected/034787fd-ba3c-4baa-be4a-8c963d565ae7-kube-api-access-5jtgh\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:28.985010 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.984844    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:28.985010 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.984895    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-tls\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:28.985010 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.984928    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:28.985010 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:28.984964    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/034787fd-ba3c-4baa-be4a-8c963d565ae7-metrics-client-ca\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.086800 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.086085    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.086800 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.086139    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jtgh\" (UniqueName: \"kubernetes.io/projected/034787fd-ba3c-4baa-be4a-8c963d565ae7-kube-api-access-5jtgh\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.086800 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.086172    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.086800 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.086220    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-tls\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.086800 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.086255    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.086800 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.086283    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/034787fd-ba3c-4baa-be4a-8c963d565ae7-metrics-client-ca\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.086800 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.086350    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-grpc-tls\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.086800 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.086385    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.087545 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.087365    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/034787fd-ba3c-4baa-be4a-8c963d565ae7-metrics-client-ca\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.090948 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.090857    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.090948 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.090899    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-tls\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.091579 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.091546    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.092500 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.092174    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-grpc-tls\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.092637 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.092615    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.093511 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.093438    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/034787fd-ba3c-4baa-be4a-8c963d565ae7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.100287 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.100240    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jtgh\" (UniqueName: \"kubernetes.io/projected/034787fd-ba3c-4baa-be4a-8c963d565ae7-kube-api-access-5jtgh\") pod \"thanos-querier-6969448c8b-z8cjs\" (UID: \"034787fd-ba3c-4baa-be4a-8c963d565ae7\") " pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.248628 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.248592    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:29.724743 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:29.724704    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hd5jl"
Jul 15 13:52:31.270127 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.270033    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6c9554f5c-ljcbm"]
Jul 15 13:52:31.272447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.272420    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.277891 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.277695    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 15 13:52:31.279013 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.278979    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7qfctqqprqf4e\""
Jul 15 13:52:31.279286 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.279136    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 15 13:52:31.279656 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.279635    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 13:52:31.280087 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.279983    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 15 13:52:31.282608 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.281344    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5lkzh\""
Jul 15 13:52:31.292887 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.292822    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c9554f5c-ljcbm"]
Jul 15 13:52:31.306678 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.306561    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb4f8\" (UniqueName: \"kubernetes.io/projected/29019267-0a03-47ef-937f-adf6b7a0ee97-kube-api-access-sb4f8\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.306678 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.306607    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-client-ca-bundle\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.306980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.306693    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29019267-0a03-47ef-937f-adf6b7a0ee97-audit-log\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.306980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.306749    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-secret-metrics-server-tls\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.306980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.306786    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-secret-metrics-server-client-certs\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.306980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.306859    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29019267-0a03-47ef-937f-adf6b7a0ee97-metrics-server-audit-profiles\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.306980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.306910    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29019267-0a03-47ef-937f-adf6b7a0ee97-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.407759 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.407717    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb4f8\" (UniqueName: \"kubernetes.io/projected/29019267-0a03-47ef-937f-adf6b7a0ee97-kube-api-access-sb4f8\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.407976 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.407771    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-client-ca-bundle\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.407976 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.407817    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29019267-0a03-47ef-937f-adf6b7a0ee97-audit-log\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.407976 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.407847    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-secret-metrics-server-tls\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.408154 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.408058    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-secret-metrics-server-client-certs\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.408154 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.408104    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29019267-0a03-47ef-937f-adf6b7a0ee97-metrics-server-audit-profiles\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.408154 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.408131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29019267-0a03-47ef-937f-adf6b7a0ee97-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.408318 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.408295    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29019267-0a03-47ef-937f-adf6b7a0ee97-audit-log\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.408879 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.408842    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29019267-0a03-47ef-937f-adf6b7a0ee97-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.409230 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.409191    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29019267-0a03-47ef-937f-adf6b7a0ee97-metrics-server-audit-profiles\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.410741 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.410699    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-client-ca-bundle\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.411172 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.411128    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-secret-metrics-server-tls\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.411548 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.411530    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29019267-0a03-47ef-937f-adf6b7a0ee97-secret-metrics-server-client-certs\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.419454 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.419422    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb4f8\" (UniqueName: \"kubernetes.io/projected/29019267-0a03-47ef-937f-adf6b7a0ee97-kube-api-access-sb4f8\") pod \"metrics-server-6c9554f5c-ljcbm\" (UID: \"29019267-0a03-47ef-937f-adf6b7a0ee97\") " pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.584750 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.584715    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:31.664266 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.663050    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"]
Jul 15 13:52:31.665461 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.665431    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:31.668850 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.668819    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 15 13:52:31.669145 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.669125    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fxclk\""
Jul 15 13:52:31.679181 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.679102    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"]
Jul 15 13:52:31.710594 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.710556    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htlpw\" (UniqueName: \"kubernetes.io/projected/0243e1da-7793-4a84-bad5-7a19983cea44-kube-api-access-htlpw\") pod \"monitoring-plugin-5f68df7f58-x6srl\" (UID: \"0243e1da-7793-4a84-bad5-7a19983cea44\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:31.710777 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.710621    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0243e1da-7793-4a84-bad5-7a19983cea44-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-x6srl\" (UID: \"0243e1da-7793-4a84-bad5-7a19983cea44\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:31.811762 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.811718    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0243e1da-7793-4a84-bad5-7a19983cea44-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-x6srl\" (UID: \"0243e1da-7793-4a84-bad5-7a19983cea44\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:31.811944 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.811812    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htlpw\" (UniqueName: \"kubernetes.io/projected/0243e1da-7793-4a84-bad5-7a19983cea44-kube-api-access-htlpw\") pod \"monitoring-plugin-5f68df7f58-x6srl\" (UID: \"0243e1da-7793-4a84-bad5-7a19983cea44\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:31.814743 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.814709    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0243e1da-7793-4a84-bad5-7a19983cea44-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-x6srl\" (UID: \"0243e1da-7793-4a84-bad5-7a19983cea44\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:31.822067 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.822038    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htlpw\" (UniqueName: \"kubernetes.io/projected/0243e1da-7793-4a84-bad5-7a19983cea44-kube-api-access-htlpw\") pod \"monitoring-plugin-5f68df7f58-x6srl\" (UID: \"0243e1da-7793-4a84-bad5-7a19983cea44\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:31.978205 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:31.978095    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:32.217956 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.217921    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"]
Jul 15 13:52:32.223365 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.222634    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.226982 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.226873    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 15 13:52:32.226982 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.226924    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 15 13:52:32.227187 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.227014    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9qqnw\""
Jul 15 13:52:32.227187 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.227146    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 15 13:52:32.229656 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.229430    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 15 13:52:32.229656 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.229475    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 15 13:52:32.234307 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.234285    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 15 13:52:32.240327 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.240301    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"]
Jul 15 13:52:32.317262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.317011    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.317262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.317068    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2xxq\" (UniqueName: \"kubernetes.io/projected/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-kube-api-access-p2xxq\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.317262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.317112    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-telemeter-client-tls\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.317262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.317177    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-secret-telemeter-client\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.317262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.317202    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-metrics-client-ca\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.317945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.317270    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.317945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.317315    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-serving-certs-ca-bundle\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.317945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.317370    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-federate-client-tls\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.418316 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.418278    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-secret-telemeter-client\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.418316 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.418324    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-metrics-client-ca\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.418568 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.418373    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.418568 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.418391    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-serving-certs-ca-bundle\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.418568 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.418413    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-federate-client-tls\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.418568 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.418456    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.418568 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.418477    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2xxq\" (UniqueName: \"kubernetes.io/projected/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-kube-api-access-p2xxq\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.418568 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.418501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-telemeter-client-tls\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.419358 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.419304    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-serving-certs-ca-bundle\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.420228 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.420201    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-metrics-client-ca\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.420357 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.420318    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.422766 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.422741    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-federate-client-tls\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.422891 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.422874    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-telemeter-client-tls\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.423089 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.423066    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-secret-telemeter-client\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.425404 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.425360    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.427498 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.427471    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2xxq\" (UniqueName: \"kubernetes.io/projected/ac0b44cc-0118-4bcc-8473-b3000b5e3ec4-kube-api-access-p2xxq\") pod \"telemeter-client-55fcc86d98-jgmtk\" (UID: \"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4\") " pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.444351 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.444300    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:52:32.536948 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.536913    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"
Jul 15 13:52:32.666677 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:32.666620    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c04fab_220d_499e_9345_9d4cc8f79198.slice/crio-0fe2fe2e395d0fdc7888d34239c8775b1ecbd12fe10c7cc84d29607ae0daa6c2 WatchSource:0}: Error finding container 0fe2fe2e395d0fdc7888d34239c8775b1ecbd12fe10c7cc84d29607ae0daa6c2: Status 404 returned error can't find the container with id 0fe2fe2e395d0fdc7888d34239c8775b1ecbd12fe10c7cc84d29607ae0daa6c2
Jul 15 13:52:32.815713 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.815667    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerStarted","Data":"0fe2fe2e395d0fdc7888d34239c8775b1ecbd12fe10c7cc84d29607ae0daa6c2"}
Jul 15 13:52:32.895564 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.895028    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"]
Jul 15 13:52:32.950624 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.948204    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6969448c8b-z8cjs"]
Jul 15 13:52:32.950624 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.950185    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c9554f5c-ljcbm"]
Jul 15 13:52:32.976110 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:32.976076    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55fcc86d98-jgmtk"]
Jul 15 13:52:33.021454 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:33.021182    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0b44cc_0118_4bcc_8473_b3000b5e3ec4.slice/crio-e87fd4662510460db1a50266c2703977c363c9607c595f0c2e6156161dc9616e WatchSource:0}: Error finding container e87fd4662510460db1a50266c2703977c363c9607c595f0c2e6156161dc9616e: Status 404 returned error can't find the container with id e87fd4662510460db1a50266c2703977c363c9607c595f0c2e6156161dc9616e
Jul 15 13:52:33.421398 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.419386    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:52:33.424215 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.424187    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.430830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.430094    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 13:52:33.430830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.430352    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 13:52:33.430830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.430592    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 13:52:33.430830 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.430607    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 13:52:33.431125 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.430904    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 13:52:33.431125 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.431115    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 13:52:33.431653 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.431468    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 13:52:33.431653 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.431482    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 13:52:33.431653 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.431507    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-22q72\""
Jul 15 13:52:33.431884 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.431685    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4l44sb0jgdiub\""
Jul 15 13:52:33.434164 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.433326    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 13:52:33.434164 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.433483    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 13:52:33.434164 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.433572    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 13:52:33.434164 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.433683    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 13:52:33.436420 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.436396    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 13:52:33.440495 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.440468    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 13:52:33.462405 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.458987    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:52:33.537465 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537427    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537627 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537478    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537627 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537550    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537627 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537594    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537795 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537642    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537795 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537670    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537795 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537694    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsssk\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-kube-api-access-hsssk\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537795 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537718    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537795 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537748    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.537795 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537772    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537809    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537878    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537919    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-config\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.537946    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.538025    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.538052    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538075 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.538074    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538462 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.538124    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.538462 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.538148    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639309    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639376    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639401    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsssk\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-kube-api-access-hsssk\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639424    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639449    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639471    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639494    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639525    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639581    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639607    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-config\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639683    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639766    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639807    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639828    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640950 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639865    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640950 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639885    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640950 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639926    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640950 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639947    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640950 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.639974    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640950 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.640001    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.640950 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.640779    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.642418 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.642391    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.645587 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.645559    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.646028 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.645998    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.646614 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.646593    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.647532 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.647488    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.649412 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.649059    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.650674 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.650371    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.654433 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.654400    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.655099 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.655073    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.655637 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.655615    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.659038 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.657757    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.659038 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.658155    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.659038 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.658542    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-config\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.659038 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.658661    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.659038 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.658678    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.659038 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.658872    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.663660 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.663627    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.664925 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.664875    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsssk\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-kube-api-access-hsssk\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.665629 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.665584    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.747265 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.746884    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:52:33.841040 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.840433    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zbg6k" event={"ID":"7cee1a78-4f58-439f-8c26-66f336634a49","Type":"ContainerStarted","Data":"549a8cc5ce43f9a0fa580f82af4570c937a8da6fb2642a60375cb66179faf979"}
Jul 15 13:52:33.841040 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.840929    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:52:33.844443 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.844220    2573 generic.go:356] "Generic (PLEG): container finished" podID="4c6a1483-128b-4350-bf4e-11782be7ae88" containerID="bceca3db55b0f70dd4d7f587291df513d0aa95291d087538150ba98c52f4ec97" exitCode=0
Jul 15 13:52:33.844897 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.844697    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lh55b" event={"ID":"4c6a1483-128b-4350-bf4e-11782be7ae88","Type":"ContainerDied","Data":"bceca3db55b0f70dd4d7f587291df513d0aa95291d087538150ba98c52f4ec97"}
Jul 15 13:52:33.852608 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.851894    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qnhlv" event={"ID":"6edabe15-150f-4758-8b05-a9c0f1b266bf","Type":"ContainerStarted","Data":"54cc7d8dfcbe8a648cd2746156342685010c2a2f424d774b0c4c452a33e06e99"}
Jul 15 13:52:33.856748 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.856639    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm" event={"ID":"29019267-0a03-47ef-937f-adf6b7a0ee97","Type":"ContainerStarted","Data":"9b04a53bbd7e762c48d9116d931a000738145ffcc6302b01752078a7b37f7589"}
Jul 15 13:52:33.864419 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.864318    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs" event={"ID":"034787fd-ba3c-4baa-be4a-8c963d565ae7","Type":"ContainerStarted","Data":"f47cfd14dee362f3faf84d80f42d22ea207a0ccd98839618c87c3c71dec72741"}
Jul 15 13:52:33.871158 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.871082    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m" event={"ID":"84d5d1ec-0377-44a4-a602-a27b899f346b","Type":"ContainerStarted","Data":"143ada3ed823e30703a7c64f106da7a37a268d29ae9213b643f8588d9e25e858"}
Jul 15 13:52:33.880052 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.879747    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp" event={"ID":"15e901bb-7e01-4872-b122-7782c92e4177","Type":"ContainerStarted","Data":"358df0da4c5a3895f6194735c95aedcf517c60cf601e890d82a17ee7505d879d"}
Jul 15 13:52:33.880052 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.879819    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp" event={"ID":"15e901bb-7e01-4872-b122-7782c92e4177","Type":"ContainerStarted","Data":"95b9f7ef4ce4949f5a292c5730a7c8452014207ac68122cf1224d8ccfec7aeba"}
Jul 15 13:52:33.880052 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.879847    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp" event={"ID":"15e901bb-7e01-4872-b122-7782c92e4177","Type":"ContainerStarted","Data":"9c1946bb720b4ceb490b41e835515b1033dab777692445bf782b41c08007d9df"}
Jul 15 13:52:33.884049 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.883668    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk" event={"ID":"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4","Type":"ContainerStarted","Data":"e87fd4662510460db1a50266c2703977c363c9607c595f0c2e6156161dc9616e"}
Jul 15 13:52:33.885040 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.884586    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zbg6k" podStartSLOduration=68.32989593 podStartE2EDuration="1m14.884565131s" totalImagesPullingTime="6.554669201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:33.862945268 +0000 UTC m=+74.982750865" watchObservedRunningTime="2026-07-15 13:52:33.884565131 +0000 UTC m=+75.004370713"
Jul 15 13:52:33.885040 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.884706    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qnhlv" podStartSLOduration=66.577331454 podStartE2EDuration="1m14.884697622s" totalImagesPullingTime="8.307366168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:33.883926123 +0000 UTC m=+75.003731702" watchObservedRunningTime="2026-07-15 13:52:33.884697622 +0000 UTC m=+75.004503184"
Jul 15 13:52:33.890208 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.890176    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl" event={"ID":"0243e1da-7793-4a84-bad5-7a19983cea44","Type":"ContainerStarted","Data":"64cf6f81e49d40f4054d0c8348f4cc819c04f5964852cb08095e6464a363c916"}
Jul 15 13:52:33.965893 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:33.965836    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-x8xjp" podStartSLOduration=3.197981506 podStartE2EDuration="7.965812687s" totalImagesPullingTime="4.767831181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:33.960366139 +0000 UTC m=+75.080171713" watchObservedRunningTime="2026-07-15 13:52:33.965812687 +0000 UTC m=+75.085618268"
Jul 15 13:52:34.007171 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:34.005937    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:52:34.011931 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:52:34.011895    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a50629f_25e1_41f2_bf43_349615478b53.slice/crio-fab89a3cf14f391aec7e18fe9d195592b2bf481cb9f18af42d775b3c107d1f4a WatchSource:0}: Error finding container fab89a3cf14f391aec7e18fe9d195592b2bf481cb9f18af42d775b3c107d1f4a: Status 404 returned error can't find the container with id fab89a3cf14f391aec7e18fe9d195592b2bf481cb9f18af42d775b3c107d1f4a
Jul 15 13:52:34.901366 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:34.900982    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lh55b" event={"ID":"4c6a1483-128b-4350-bf4e-11782be7ae88","Type":"ContainerStarted","Data":"378570d5d342b9dd56b1820ec9c6af069db69f3e27b08fa2b7f2c87e318ec28d"}
Jul 15 13:52:34.901366 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:34.901090    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lh55b" event={"ID":"4c6a1483-128b-4350-bf4e-11782be7ae88","Type":"ContainerStarted","Data":"86933dc235233c7836225e045d35bc4a2a31aa8552b7bd53c3199700d1851000"}
Jul 15 13:52:34.906493 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:34.905401    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerStarted","Data":"fab89a3cf14f391aec7e18fe9d195592b2bf481cb9f18af42d775b3c107d1f4a"}
Jul 15 13:52:34.929451 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:34.929389    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lh55b" podStartSLOduration=4.356572455 podStartE2EDuration="8.929364888s" totalImagesPullingTime="4.419984722s" totalInitContainerRuntime="152.807711ms" isStatefulPod=true podCreationTimestamp="2026-07-15 13:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:34.928118373 +0000 UTC m=+76.047923966" watchObservedRunningTime="2026-07-15 13:52:34.929364888 +0000 UTC m=+76.049170468"
Jul 15 13:52:34.931304 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:34.930918    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-7ss4m" podStartSLOduration=4.273801384 podStartE2EDuration="8.930896128s" totalImagesPullingTime="4.657094744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:34.011417484 +0000 UTC m=+75.131223066" watchObservedRunningTime="2026-07-15 13:52:34.930896128 +0000 UTC m=+76.050701709"
Jul 15 13:52:37.921295 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.921108    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs" event={"ID":"034787fd-ba3c-4baa-be4a-8c963d565ae7","Type":"ContainerStarted","Data":"3cdc9391dde34af102658d864e0f213d5ce9cc1df84d272c61dfb86f1367d149"}
Jul 15 13:52:37.921295 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.921198    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs" event={"ID":"034787fd-ba3c-4baa-be4a-8c963d565ae7","Type":"ContainerStarted","Data":"50be0921b7d2015f6e138416085cf9cd58beaa7f2cf2f74bfbf48cc718fe00df"}
Jul 15 13:52:37.921295 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.921241    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs" event={"ID":"034787fd-ba3c-4baa-be4a-8c963d565ae7","Type":"ContainerStarted","Data":"8c30d7c777853daaeedde005bf050ece4a8c6c3bf43ea260ca658859cfeb7d4d"}
Jul 15 13:52:37.924712 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.924678    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk" event={"ID":"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4","Type":"ContainerStarted","Data":"b1f0ed5e0ebd753155c7281b638636853e19bc1545126ad66763cf616176bda6"}
Jul 15 13:52:37.924862 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.924745    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk" event={"ID":"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4","Type":"ContainerStarted","Data":"8ec015e9e0140e43e4c17147e4bcdabc75748e996c42dc375a9182295af36b69"}
Jul 15 13:52:37.924862 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.924773    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk" event={"ID":"ac0b44cc-0118-4bcc-8473-b3000b5e3ec4","Type":"ContainerStarted","Data":"f4fcdc3ebd3ddec75c0cba4e8ab059dc5a1e63d607d75929ad1cfa692f2c6cc4"}
Jul 15 13:52:37.927222 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.927126    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a50629f-25e1-41f2-bf43-349615478b53" containerID="4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65" exitCode=0
Jul 15 13:52:37.927388 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.927272    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerDied","Data":"4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65"}
Jul 15 13:52:37.929871 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.929842    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm" event={"ID":"29019267-0a03-47ef-937f-adf6b7a0ee97","Type":"ContainerStarted","Data":"90722777cde462dbe706b2467616650c98dff305486a45f7939af80915ca1a82"}
Jul 15 13:52:37.931815 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.931788    2573 generic.go:356] "Generic (PLEG): container finished" podID="68c04fab-220d-499e-9345-9d4cc8f79198" containerID="7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace" exitCode=0
Jul 15 13:52:37.931904 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.931821    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerDied","Data":"7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace"}
Jul 15 13:52:37.933808 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.933780    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl" event={"ID":"0243e1da-7793-4a84-bad5-7a19983cea44","Type":"ContainerStarted","Data":"a01ebe07da4103a5e4fcf062eb9948705f119a023bb324639490335c0f8eff3c"}
Jul 15 13:52:37.934111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.934049    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:37.940430 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.940375    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl"
Jul 15 13:52:37.949720 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.949635    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55fcc86d98-jgmtk" podStartSLOduration=1.9922764069999999 podStartE2EDuration="5.949614008s" totalImagesPullingTime="3.957337601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:37.949232455 +0000 UTC m=+79.069038068" watchObservedRunningTime="2026-07-15 13:52:37.949614008 +0000 UTC m=+79.069419589"
Jul 15 13:52:37.972697 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:37.972636    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x6srl" podStartSLOduration=2.88691891 podStartE2EDuration="6.972621357s" totalImagesPullingTime="4.085702447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:37.970767463 +0000 UTC m=+79.090573043" watchObservedRunningTime="2026-07-15 13:52:37.972621357 +0000 UTC m=+79.092426930"
Jul 15 13:52:38.060702 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:38.060645    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm" podStartSLOduration=3.042943055 podStartE2EDuration="7.060625873s" totalImagesPullingTime="4.017682818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:38.059896958 +0000 UTC m=+79.179702541" watchObservedRunningTime="2026-07-15 13:52:38.060625873 +0000 UTC m=+79.180431431"
Jul 15 13:52:45.974053 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:45.973954    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs" event={"ID":"034787fd-ba3c-4baa-be4a-8c963d565ae7","Type":"ContainerStarted","Data":"110bb91dcd0cec85968946c2030c0f66bec0576a2cb4b271a6954d657a6e9d02"}
Jul 15 13:52:45.974566 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:45.974050    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs" event={"ID":"034787fd-ba3c-4baa-be4a-8c963d565ae7","Type":"ContainerStarted","Data":"cc9ee6b15135ae42aa1d89d6c97fa55bd1ef714b63f242915d582804e18bba70"}
Jul 15 13:52:45.974566 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:45.974089    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs" event={"ID":"034787fd-ba3c-4baa-be4a-8c963d565ae7","Type":"ContainerStarted","Data":"628bf107e6d548cfaf66cef54a6d56f1fae4f24488024cc29b9736861ad624ef"}
Jul 15 13:52:45.974566 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:45.974367    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:45.981014 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:45.980927    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs"
Jul 15 13:52:46.001097 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:46.001031    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6969448c8b-z8cjs" podStartSLOduration=6.01944114 podStartE2EDuration="18.001010035s" totalImagesPullingTime="11.981568895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:45.998834332 +0000 UTC m=+87.118639912" watchObservedRunningTime="2026-07-15 13:52:46.001010035 +0000 UTC m=+87.120815805"
Jul 15 13:52:46.990411 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:46.990370    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-dx9lz" event={"ID":"e7fe77c2-bfaf-4163-83b8-24a2cba8b380","Type":"ContainerStarted","Data":"48b83bc9895ec3e6b51e978b25dfb5845b92b2187b6e73f0ecbf76038e41089c"}
Jul 15 13:52:46.991200 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:46.991137    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-dx9lz"
Jul 15 13:52:47.003317 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:47.003251    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-dx9lz"
Jul 15 13:52:47.014118 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:47.013369    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-dx9lz" podStartSLOduration=2.219521646 podStartE2EDuration="24.013346019s" totalImagesPullingTime="21.793824373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:47.013320228 +0000 UTC m=+88.133125804" watchObservedRunningTime="2026-07-15 13:52:47.013346019 +0000 UTC m=+88.133151594"
Jul 15 13:52:47.998263 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:47.997655    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerStarted","Data":"a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9"}
Jul 15 13:52:47.998263 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:47.997723    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerStarted","Data":"fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833"}
Jul 15 13:52:50.019271 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:50.019226    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerStarted","Data":"d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f"}
Jul 15 13:52:50.024066 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:50.024030    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerStarted","Data":"fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673"}
Jul 15 13:52:50.024216 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:50.024115    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerStarted","Data":"9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594"}
Jul 15 13:52:50.024216 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:50.024157    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerStarted","Data":"1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961"}
Jul 15 13:52:50.024216 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:50.024197    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerStarted","Data":"6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0"}
Jul 15 13:52:50.064789 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:50.063622    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.918423144 podStartE2EDuration="23.063598884s" totalImagesPullingTime="14.104321784s" totalInitContainerRuntime="40.853956ms" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:50.059181649 +0000 UTC m=+91.178987242" watchObservedRunningTime="2026-07-15 13:52:50.063598884 +0000 UTC m=+91.183404466"
Jul 15 13:52:51.034447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:51.034404    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerStarted","Data":"c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930"}
Jul 15 13:52:51.034893 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:51.034498    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerStarted","Data":"84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1"}
Jul 15 13:52:51.034893 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:51.034541    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerStarted","Data":"e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040"}
Jul 15 13:52:51.034893 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:51.034571    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerStarted","Data":"00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27"}
Jul 15 13:52:51.034893 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:51.034596    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerStarted","Data":"aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03"}
Jul 15 13:52:51.072077 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:51.072014    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.175521514 podStartE2EDuration="18.071988796s" totalImagesPullingTime="14.860903672s" totalInitContainerRuntime="35.56361ms" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:51.069645197 +0000 UTC m=+92.189450776" watchObservedRunningTime="2026-07-15 13:52:51.071988796 +0000 UTC m=+92.191794376"
Jul 15 13:52:51.585732 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:51.585695    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:51.585950 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:51.585766    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:52:53.748375 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:52:53.748323    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:04.908377 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:04.908347    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zbg6k"
Jul 15 13:53:11.590541 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:11.590510    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:53:11.594281 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:11.594255    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6c9554f5c-ljcbm"
Jul 15 13:53:33.747677 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:33.747635    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:33.810589 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:33.810561    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:34.240446 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:34.240413    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:47.302157 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:47.302100    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:53:47.302657 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:47.302515    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="alertmanager" containerID="cri-o://fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833" gracePeriod=120
Jul 15 13:53:47.302657 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:47.302563    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy-metric" containerID="cri-o://9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594" gracePeriod=120
Jul 15 13:53:47.302657 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:47.302563    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="prom-label-proxy" containerID="cri-o://fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673" gracePeriod=120
Jul 15 13:53:47.302657 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:47.302575    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy-web" containerID="cri-o://6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0" gracePeriod=120
Jul 15 13:53:47.302657 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:47.302608    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="config-reloader" containerID="cri-o://a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9" gracePeriod=120
Jul 15 13:53:47.302657 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:47.302641    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy" containerID="cri-o://1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961" gracePeriod=120
Jul 15 13:53:48.282792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282756    2573 generic.go:356] "Generic (PLEG): container finished" podID="68c04fab-220d-499e-9345-9d4cc8f79198" containerID="fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673" exitCode=0
Jul 15 13:53:48.282792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282780    2573 generic.go:356] "Generic (PLEG): container finished" podID="68c04fab-220d-499e-9345-9d4cc8f79198" containerID="9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594" exitCode=0
Jul 15 13:53:48.282792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282787    2573 generic.go:356] "Generic (PLEG): container finished" podID="68c04fab-220d-499e-9345-9d4cc8f79198" containerID="1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961" exitCode=0
Jul 15 13:53:48.282792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282792    2573 generic.go:356] "Generic (PLEG): container finished" podID="68c04fab-220d-499e-9345-9d4cc8f79198" containerID="a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9" exitCode=0
Jul 15 13:53:48.282792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282797    2573 generic.go:356] "Generic (PLEG): container finished" podID="68c04fab-220d-499e-9345-9d4cc8f79198" containerID="fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833" exitCode=0
Jul 15 13:53:48.283077 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282828    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerDied","Data":"fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673"}
Jul 15 13:53:48.283077 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282899    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerDied","Data":"9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594"}
Jul 15 13:53:48.283077 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282925    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerDied","Data":"1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961"}
Jul 15 13:53:48.283077 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282949    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerDied","Data":"a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9"}
Jul 15 13:53:48.283077 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.282974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerDied","Data":"fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833"}
Jul 15 13:53:48.548105 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.548078    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:48.648569 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.648532    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.648569 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.648576    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-web-config\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.648816 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.648596    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-config-volume\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.648816 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.648623    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-cluster-tls-config\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.648816 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.648647    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-metric\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.648816 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.648674    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-tls-assets\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.648816 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.648694    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-web\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.648816 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.648729    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-trusted-ca-bundle\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.649210 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.649150    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:48.649278 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.649219    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-metrics-client-ca\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.649325 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.649292    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-main-db\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.649401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.649324    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4gs2\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-kube-api-access-p4gs2\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.649401 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.649385    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-config-out\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.649499 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.649412    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls\") pod \"68c04fab-220d-499e-9345-9d4cc8f79198\" (UID: \"68c04fab-220d-499e-9345-9d4cc8f79198\") "
Jul 15 13:53:48.650014 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.649757    2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.650014 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.649799    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:48.650470 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.650439    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:53:48.651776 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.651625    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:53:48.651876 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.651823    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-config-volume" (OuterVolumeSpecName: "config-volume") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:48.652312 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.652282    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:48.652681 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.652635    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:48.652798 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.652705    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:48.653029 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.653013    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-kube-api-access-p4gs2" (OuterVolumeSpecName: "kube-api-access-p4gs2") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "kube-api-access-p4gs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:53:48.653110 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.653033    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:48.653788 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.653759    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-config-out" (OuterVolumeSpecName: "config-out") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:53:48.656897 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.656805    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:48.664185 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.664153    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-web-config" (OuterVolumeSpecName: "web-config") pod "68c04fab-220d-499e-9345-9d4cc8f79198" (UID: "68c04fab-220d-499e-9345-9d4cc8f79198"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:48.750426 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750392    2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-tls-assets\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750426 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750418    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750426 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750427    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68c04fab-220d-499e-9345-9d4cc8f79198-metrics-client-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750426 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750436    2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-alertmanager-main-db\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750445    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4gs2\" (UniqueName: \"kubernetes.io/projected/68c04fab-220d-499e-9345-9d4cc8f79198-kube-api-access-p4gs2\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750454    2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68c04fab-220d-499e-9345-9d4cc8f79198-config-out\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750461    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-main-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750471    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750480    2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-web-config\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750489    2573 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-config-volume\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750496    2573 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-cluster-tls-config\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:48.750669 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:48.750504    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68c04fab-220d-499e-9345-9d4cc8f79198-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:49.294828 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.294779    2573 generic.go:356] "Generic (PLEG): container finished" podID="68c04fab-220d-499e-9345-9d4cc8f79198" containerID="6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0" exitCode=0
Jul 15 13:53:49.294828 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.294831    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerDied","Data":"6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0"}
Jul 15 13:53:49.295031 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.294875    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.295031 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.294934    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68c04fab-220d-499e-9345-9d4cc8f79198","Type":"ContainerDied","Data":"0fe2fe2e395d0fdc7888d34239c8775b1ecbd12fe10c7cc84d29607ae0daa6c2"}
Jul 15 13:53:49.295031 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.294982    2573 scope.go:117] "RemoveContainer" containerID="fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673"
Jul 15 13:53:49.303375 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.303353    2573 scope.go:117] "RemoveContainer" containerID="9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594"
Jul 15 13:53:49.311369 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.311348    2573 scope.go:117] "RemoveContainer" containerID="1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961"
Jul 15 13:53:49.318322 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.318304    2573 scope.go:117] "RemoveContainer" containerID="6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0"
Jul 15 13:53:49.320857 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.320835    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:53:49.326515 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.326325    2573 scope.go:117] "RemoveContainer" containerID="a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9"
Jul 15 13:53:49.329530 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.329506    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:53:49.335193 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.335172    2573 scope.go:117] "RemoveContainer" containerID="fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833"
Jul 15 13:53:49.342892 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.342872    2573 scope.go:117] "RemoveContainer" containerID="7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace"
Jul 15 13:53:49.350478 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.350460    2573 scope.go:117] "RemoveContainer" containerID="fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673"
Jul 15 13:53:49.350966 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:49.350935    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673\": container with ID starting with fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673 not found: ID does not exist" containerID="fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673"
Jul 15 13:53:49.351086 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.350979    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673"} err="failed to get container status \"fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673\": rpc error: code = NotFound desc = could not find container \"fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673\": container with ID starting with fc9e59aed74e92d940ebf221a5c544bddc7f009adb64380f568d1b66c17ce673 not found: ID does not exist"
Jul 15 13:53:49.351086 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.351025    2573 scope.go:117] "RemoveContainer" containerID="9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594"
Jul 15 13:53:49.351527 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:49.351500    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594\": container with ID starting with 9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594 not found: ID does not exist" containerID="9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594"
Jul 15 13:53:49.351607 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.351531    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594"} err="failed to get container status \"9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594\": rpc error: code = NotFound desc = could not find container \"9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594\": container with ID starting with 9ea3953b14469d2191fb221aa6f9b80a33c5cf2f9b61dc93371e3b64f03ca594 not found: ID does not exist"
Jul 15 13:53:49.351607 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.351549    2573 scope.go:117] "RemoveContainer" containerID="1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961"
Jul 15 13:53:49.351810 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:49.351791    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961\": container with ID starting with 1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961 not found: ID does not exist" containerID="1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961"
Jul 15 13:53:49.351863 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.351817    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961"} err="failed to get container status \"1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961\": rpc error: code = NotFound desc = could not find container \"1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961\": container with ID starting with 1543a42cb5653d94cf66a354c56418ec6643cfb6f82d8103e6df6a664f5b5961 not found: ID does not exist"
Jul 15 13:53:49.351863 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.351834    2573 scope.go:117] "RemoveContainer" containerID="6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0"
Jul 15 13:53:49.352070 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:49.352048    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0\": container with ID starting with 6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0 not found: ID does not exist" containerID="6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0"
Jul 15 13:53:49.352133 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.352076    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0"} err="failed to get container status \"6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0\": rpc error: code = NotFound desc = could not find container \"6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0\": container with ID starting with 6d2e6f3fc548f811e7be826e6da44d042719f28c821dadfda7a2201967b532f0 not found: ID does not exist"
Jul 15 13:53:49.352133 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.352092    2573 scope.go:117] "RemoveContainer" containerID="a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9"
Jul 15 13:53:49.352323 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:49.352307    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9\": container with ID starting with a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9 not found: ID does not exist" containerID="a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9"
Jul 15 13:53:49.352387 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.352327    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9"} err="failed to get container status \"a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9\": rpc error: code = NotFound desc = could not find container \"a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9\": container with ID starting with a5d4d55dfb1b83578f09daae4f39fea8b68686bd2097fc30d1d6d8eb623ebcc9 not found: ID does not exist"
Jul 15 13:53:49.352387 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.352360    2573 scope.go:117] "RemoveContainer" containerID="fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833"
Jul 15 13:53:49.352612 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:49.352587    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833\": container with ID starting with fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833 not found: ID does not exist" containerID="fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833"
Jul 15 13:53:49.352676 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.352613    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833"} err="failed to get container status \"fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833\": rpc error: code = NotFound desc = could not find container \"fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833\": container with ID starting with fdc6b4b0bf92e3a05791b149e2df2be753695470b4f6bdad45087c3e9e6e1833 not found: ID does not exist"
Jul 15 13:53:49.352676 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.352629    2573 scope.go:117] "RemoveContainer" containerID="7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace"
Jul 15 13:53:49.352840 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:49.352823    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace\": container with ID starting with 7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace not found: ID does not exist" containerID="7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace"
Jul 15 13:53:49.352893 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.352843    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace"} err="failed to get container status \"7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace\": rpc error: code = NotFound desc = could not find container \"7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace\": container with ID starting with 7911dd1ecea1842d20fcaf9f8ddc8b379a8eb89677ee623a014781b477272ace not found: ID does not exist"
Jul 15 13:53:49.365442 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365410    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:53:49.365809 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365784    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy-metric"
Jul 15 13:53:49.365809 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365804    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy-metric"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365817    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="prom-label-proxy"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365827    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="prom-label-proxy"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365843    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="config-reloader"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365851    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="config-reloader"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365864    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="init-config-reloader"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365872    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="init-config-reloader"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365893    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy-web"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365902    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy-web"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365915    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="alertmanager"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365924    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="alertmanager"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365932    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy"
Jul 15 13:53:49.365955 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.365940    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy"
Jul 15 13:53:49.366314 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.366028    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="prom-label-proxy"
Jul 15 13:53:49.366314 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.366038    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="config-reloader"
Jul 15 13:53:49.366314 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.366045    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="alertmanager"
Jul 15 13:53:49.366314 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.366053    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy-web"
Jul 15 13:53:49.366314 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.366059    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy-metric"
Jul 15 13:53:49.366314 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.366066    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" containerName="kube-rbac-proxy"
Jul 15 13:53:49.370174 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.370154    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.374471 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.374442    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 15 13:53:49.374610 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.374442    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 15 13:53:49.374610 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.374443    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 15 13:53:49.374731 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.374683    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 15 13:53:49.374731 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.374708    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pb8lv\""
Jul 15 13:53:49.374731 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.374717    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 15 13:53:49.374874 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.374781    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 15 13:53:49.375200 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.375185    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 15 13:53:49.375544 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.375531    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 15 13:53:49.381855 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.381828    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 15 13:53:49.399968 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.399935    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:53:49.405996 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.405965    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68c04fab-220d-499e-9345-9d4cc8f79198" path="/var/lib/kubelet/pods/68c04fab-220d-499e-9345-9d4cc8f79198/volumes"
Jul 15 13:53:49.456610 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456571    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvxw2\" (UniqueName: \"kubernetes.io/projected/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-kube-api-access-wvxw2\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456624    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456650    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456683    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456716    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456739    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-config-volume\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456969 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456822    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456969 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456872    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-web-config\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456969 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456928    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.456969 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.456960    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.457125 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.457021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-config-out\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.457125 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.457047    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.457125 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.457073    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558461 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558374    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvxw2\" (UniqueName: \"kubernetes.io/projected/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-kube-api-access-wvxw2\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558461 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558422    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558461 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558445    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558995 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558478    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558995 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558515    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558995 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558548    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-config-volume\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558995 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558995 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558749    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-web-config\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558995 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558797    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558995 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558831    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.558995 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.558863    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-config-out\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.559376 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.559084    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.559376 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.559121    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.559560 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.559509    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.560138 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.560112    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.561469 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.561327    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.561469 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.561398    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-config-volume\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.561635 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.561618    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.561801 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.561778    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.561909 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.561785    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.562262 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.562241    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.562315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.562297    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.562431 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.562413    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.562723 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.562702    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-config-out\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.563464 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.563446    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-web-config\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.568431 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.568407    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvxw2\" (UniqueName: \"kubernetes.io/projected/803e9481-cf13-4f7a-87e8-e6a8c93c5bc0-kube-api-access-wvxw2\") pod \"alertmanager-main-0\" (UID: \"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.688390 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.688355    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 13:53:49.829160 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:49.829134    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 13:53:49.831524 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:53:49.831499    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803e9481_cf13_4f7a_87e8_e6a8c93c5bc0.slice/crio-c44d6ef39a533ce8d8d621309f2b5d0e5ee964d3c75167d26b5d4aa658444a00 WatchSource:0}: Error finding container c44d6ef39a533ce8d8d621309f2b5d0e5ee964d3c75167d26b5d4aa658444a00: Status 404 returned error can't find the container with id c44d6ef39a533ce8d8d621309f2b5d0e5ee964d3c75167d26b5d4aa658444a00
Jul 15 13:53:50.301100 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:50.301053    2573 generic.go:356] "Generic (PLEG): container finished" podID="803e9481-cf13-4f7a-87e8-e6a8c93c5bc0" containerID="1bfbca283d3947e41cc63618f0574d87d98793c49481bd75f911eb266ee6a828" exitCode=0
Jul 15 13:53:50.301301 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:50.301130    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0","Type":"ContainerDied","Data":"1bfbca283d3947e41cc63618f0574d87d98793c49481bd75f911eb266ee6a828"}
Jul 15 13:53:50.301301 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:50.301213    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0","Type":"ContainerStarted","Data":"c44d6ef39a533ce8d8d621309f2b5d0e5ee964d3c75167d26b5d4aa658444a00"}
Jul 15 13:53:51.309801 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.309759    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0","Type":"ContainerStarted","Data":"71fda17cc408abfabe7511f56cf8d0d0c1f7a80d73cc5df42bd326402b8df54f"}
Jul 15 13:53:51.310218 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.309879    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0","Type":"ContainerStarted","Data":"04a11fa45d31a6a42db23761ffde59b1f7371c2a8fdf57e4b82811d961ca095f"}
Jul 15 13:53:51.310218 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.309921    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0","Type":"ContainerStarted","Data":"674ff6d56c4f43b645835d7e5fae1a88736f2694fbf98d38d7594ba0ac71c5ab"}
Jul 15 13:53:51.310218 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.309954    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0","Type":"ContainerStarted","Data":"15cb6c012a2d83a1248b29e3d8923d8b9e9a0b92c1ee7d31d0ac9926f7c7e3ed"}
Jul 15 13:53:51.310218 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.309988    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0","Type":"ContainerStarted","Data":"f42d35110f62042c472ad3626b17cf8c82d551335bc6489ac7135bd825d41488"}
Jul 15 13:53:51.310218 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.310026    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"803e9481-cf13-4f7a-87e8-e6a8c93c5bc0","Type":"ContainerStarted","Data":"49d739b9e055ea66529446cf2e6cb846b34a6cf2c2affd4fdf5d74183694baa2"}
Jul 15 13:53:51.364986 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.364912    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.351172285 podStartE2EDuration="2.364888397s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.716112ms" isStatefulPod=false podCreationTimestamp="2026-07-15 13:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:53:51.358321681 +0000 UTC m=+152.478127262" watchObservedRunningTime="2026-07-15 13:53:51.364888397 +0000 UTC m=+152.484693978"
Jul 15 13:53:51.550111 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.550063    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:53:51.550430 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.550392    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="prometheus" containerID="cri-o://d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f" gracePeriod=600
Jul 15 13:53:51.550576 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.550416    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy" containerID="cri-o://84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1" gracePeriod=600
Jul 15 13:53:51.550576 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.550444    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930" gracePeriod=600
Jul 15 13:53:51.550576 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.550447    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="thanos-sidecar" containerID="cri-o://00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27" gracePeriod=600
Jul 15 13:53:51.550576 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.550469    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy-web" containerID="cri-o://e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040" gracePeriod=600
Jul 15 13:53:51.550576 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:51.550449    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="config-reloader" containerID="cri-o://aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03" gracePeriod=600
Jul 15 13:53:52.318348 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318298    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a50629f-25e1-41f2-bf43-349615478b53" containerID="c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930" exitCode=0
Jul 15 13:53:52.318348 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318324    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a50629f-25e1-41f2-bf43-349615478b53" containerID="84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1" exitCode=0
Jul 15 13:53:52.318348 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318347    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a50629f-25e1-41f2-bf43-349615478b53" containerID="00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27" exitCode=0
Jul 15 13:53:52.318348 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318353    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a50629f-25e1-41f2-bf43-349615478b53" containerID="aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03" exitCode=0
Jul 15 13:53:52.318348 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318358    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a50629f-25e1-41f2-bf43-349615478b53" containerID="d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f" exitCode=0
Jul 15 13:53:52.318842 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318362    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerDied","Data":"c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930"}
Jul 15 13:53:52.318842 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318425    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerDied","Data":"84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1"}
Jul 15 13:53:52.318842 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318451    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerDied","Data":"00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27"}
Jul 15 13:53:52.318842 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318477    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerDied","Data":"aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03"}
Jul 15 13:53:52.318842 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.318501    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerDied","Data":"d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f"}
Jul 15 13:53:52.796268 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.796242    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:52.890890 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.890793    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-kubelet-serving-ca-bundle\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.890890 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.890839    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-0\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.890890 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.890867    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-tls-assets\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.890890 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.890891    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsssk\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-kube-api-access-hsssk\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891215 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891070    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-thanos-prometheus-http-client-file\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891215 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891104    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-metrics-client-certs\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891215 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891142    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-1\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891215 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891162    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-kube-rbac-proxy\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891215 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891194    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-web-config\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891223    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-2\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891251    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-config-out\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891257    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891285    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-tls\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891319    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-db\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891360    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-config\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891388    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-metrics-client-ca\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891433    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-grpc-tls\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891483 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891463    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891891 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891515    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891891 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891543    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-trusted-ca-bundle\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891891 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891570    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-serving-certs-ca-bundle\") pod \"3a50629f-25e1-41f2-bf43-349615478b53\" (UID: \"3a50629f-25e1-41f2-bf43-349615478b53\") "
Jul 15 13:53:52.891891 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891760    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:52.891891 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891881    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.892129 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.891899    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.892129 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.892060    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:52.892491 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.892459    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:52.894836 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.892470    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:52.894836 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.893863    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:52.894836 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.894481    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:53:52.895058 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.894856    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:53:52.895285 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.895250    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.896534 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.895288    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-kube-api-access-hsssk" (OuterVolumeSpecName: "kube-api-access-hsssk") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "kube-api-access-hsssk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:53:52.896534 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.895403    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.899386 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.897953    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.901484 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.899982    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:53:52.902321 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.902047    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.902412 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.902371    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-config-out" (OuterVolumeSpecName: "config-out") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:53:52.902412 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.902385    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.902523 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.902505    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.903745 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.903718    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.904009 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.903987    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-config" (OuterVolumeSpecName: "config") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.911093 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.911060    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-web-config" (OuterVolumeSpecName: "web-config") pod "3a50629f-25e1-41f2-bf43-349615478b53" (UID: "3a50629f-25e1-41f2-bf43-349615478b53"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:53:52.993321 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993279    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993321 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993314    2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-tls-assets\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993321 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993327    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsssk\" (UniqueName: \"kubernetes.io/projected/3a50629f-25e1-41f2-bf43-349615478b53-kube-api-access-hsssk\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993373    2573 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993387    2573 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-metrics-client-certs\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993398    2573 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-kube-rbac-proxy\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993436    2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-web-config\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993448    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993460    2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-config-out\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993471    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993482    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-k8s-db\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993496    2573 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-config\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993507    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-metrics-client-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993521    2573 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-grpc-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993533    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993548    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a50629f-25e1-41f2-bf43-349615478b53-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993562    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:52.993632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:52.993575    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a50629f-25e1-41f2-bf43-349615478b53-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:53:53.325662 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.325626    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a50629f-25e1-41f2-bf43-349615478b53" containerID="e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040" exitCode=0
Jul 15 13:53:53.326057 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.325690    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerDied","Data":"e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040"}
Jul 15 13:53:53.326057 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.325721    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.326057 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.325773    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a50629f-25e1-41f2-bf43-349615478b53","Type":"ContainerDied","Data":"fab89a3cf14f391aec7e18fe9d195592b2bf481cb9f18af42d775b3c107d1f4a"}
Jul 15 13:53:53.326057 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.325805    2573 scope.go:117] "RemoveContainer" containerID="c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930"
Jul 15 13:53:53.334125 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.334104    2573 scope.go:117] "RemoveContainer" containerID="84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1"
Jul 15 13:53:53.342026 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.341998    2573 scope.go:117] "RemoveContainer" containerID="e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040"
Jul 15 13:53:53.349684 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.349657    2573 scope.go:117] "RemoveContainer" containerID="00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27"
Jul 15 13:53:53.351676 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.351646    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:53:53.355870 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.355839    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:53:53.358173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.358148    2573 scope.go:117] "RemoveContainer" containerID="aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03"
Jul 15 13:53:53.365458 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.365440    2573 scope.go:117] "RemoveContainer" containerID="d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f"
Jul 15 13:53:53.373676 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.373658    2573 scope.go:117] "RemoveContainer" containerID="4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65"
Jul 15 13:53:53.381301 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.381280    2573 scope.go:117] "RemoveContainer" containerID="c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930"
Jul 15 13:53:53.381598 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:53.381579    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930\": container with ID starting with c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930 not found: ID does not exist" containerID="c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930"
Jul 15 13:53:53.381651 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.381608    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930"} err="failed to get container status \"c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930\": rpc error: code = NotFound desc = could not find container \"c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930\": container with ID starting with c01fa7d6bbef8a320f72e97dc9907d51c1528efe8293e7e47b2eb5c552fca930 not found: ID does not exist"
Jul 15 13:53:53.381651 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.381627    2573 scope.go:117] "RemoveContainer" containerID="84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1"
Jul 15 13:53:53.381835 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:53.381820    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1\": container with ID starting with 84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1 not found: ID does not exist" containerID="84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1"
Jul 15 13:53:53.381898 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.381841    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1"} err="failed to get container status \"84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1\": rpc error: code = NotFound desc = could not find container \"84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1\": container with ID starting with 84de5d47278c175f95bd8c51f83576b3e2086b63da12d8b1241bcf41b69357f1 not found: ID does not exist"
Jul 15 13:53:53.381898 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.381854    2573 scope.go:117] "RemoveContainer" containerID="e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040"
Jul 15 13:53:53.382048 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:53.382034    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040\": container with ID starting with e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040 not found: ID does not exist" containerID="e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040"
Jul 15 13:53:53.382106 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.382051    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040"} err="failed to get container status \"e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040\": rpc error: code = NotFound desc = could not find container \"e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040\": container with ID starting with e6a7f5a68484a97eb64068704ae74b82443b12e515aa3e420747347b3b280040 not found: ID does not exist"
Jul 15 13:53:53.382106 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.382062    2573 scope.go:117] "RemoveContainer" containerID="00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27"
Jul 15 13:53:53.382447 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:53.382269    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27\": container with ID starting with 00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27 not found: ID does not exist" containerID="00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27"
Jul 15 13:53:53.382447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.382287    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27"} err="failed to get container status \"00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27\": rpc error: code = NotFound desc = could not find container \"00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27\": container with ID starting with 00b41cd6dcad9897959e1864c33f1bd8733495c93ec9feb413bf07ced504ef27 not found: ID does not exist"
Jul 15 13:53:53.382447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.382298    2573 scope.go:117] "RemoveContainer" containerID="aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03"
Jul 15 13:53:53.382621 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:53.382498    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03\": container with ID starting with aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03 not found: ID does not exist" containerID="aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03"
Jul 15 13:53:53.382621 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.382517    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03"} err="failed to get container status \"aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03\": rpc error: code = NotFound desc = could not find container \"aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03\": container with ID starting with aa2f536f151f51e440a3cf12835f2b74380504210d35b27d1bdc3d1e71edec03 not found: ID does not exist"
Jul 15 13:53:53.382621 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.382529    2573 scope.go:117] "RemoveContainer" containerID="d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f"
Jul 15 13:53:53.382843 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:53.382751    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f\": container with ID starting with d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f not found: ID does not exist" containerID="d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f"
Jul 15 13:53:53.382843 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.382781    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f"} err="failed to get container status \"d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f\": rpc error: code = NotFound desc = could not find container \"d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f\": container with ID starting with d62e2c1fe7bb378b9126e9062e261c340898489d155e82c7a1bfe71e32900e5f not found: ID does not exist"
Jul 15 13:53:53.382843 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.382806    2573 scope.go:117] "RemoveContainer" containerID="4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65"
Jul 15 13:53:53.383103 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:53:53.383025    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65\": container with ID starting with 4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65 not found: ID does not exist" containerID="4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65"
Jul 15 13:53:53.383103 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.383047    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65"} err="failed to get container status \"4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65\": rpc error: code = NotFound desc = could not find container \"4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65\": container with ID starting with 4dea1f9ea2e93afb69675e3a66aa0cabc483e6ec10bb102e2321abfee91bfb65 not found: ID does not exist"
Jul 15 13:53:53.386028 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386002    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:53:53.386357 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386325    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy-web"
Jul 15 13:53:53.386413 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386359    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy-web"
Jul 15 13:53:53.386413 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386372    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="thanos-sidecar"
Jul 15 13:53:53.386413 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386377    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="thanos-sidecar"
Jul 15 13:53:53.386413 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386391    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="init-config-reloader"
Jul 15 13:53:53.386413 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386396    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="init-config-reloader"
Jul 15 13:53:53.386413 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386408    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy-thanos"
Jul 15 13:53:53.386413 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386413    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy-thanos"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386420    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="config-reloader"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386425    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="config-reloader"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386437    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="prometheus"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386442    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="prometheus"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386450    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386455    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386502    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy-thanos"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386510    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="config-reloader"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386516    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386521    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="kube-rbac-proxy-web"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386527    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="prometheus"
Jul 15 13:53:53.386630 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.386532    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a50629f-25e1-41f2-bf43-349615478b53" containerName="thanos-sidecar"
Jul 15 13:53:53.390246 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.390226    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.393181 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.393160    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 13:53:53.393329 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.393165    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 13:53:53.393329 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.393164    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 13:53:53.394430 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.394415    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-22q72\""
Jul 15 13:53:53.394584 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.394562    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 13:53:53.394684 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.394594    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 13:53:53.394827 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.394812    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 13:53:53.394968 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.394951    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4l44sb0jgdiub\""
Jul 15 13:53:53.395022 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.394997    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 13:53:53.395117 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.395098    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 13:53:53.395243 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.395223    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 13:53:53.395348 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.395242    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 13:53:53.395348 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.395291    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 13:53:53.395777 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.395695    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 13:53:53.399774 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.399751    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 13:53:53.402149 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.402130    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 13:53:53.407134 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.407107    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a50629f-25e1-41f2-bf43-349615478b53" path="/var/lib/kubelet/pods/3a50629f-25e1-41f2-bf43-349615478b53/volumes"
Jul 15 13:53:53.409083 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.409060    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:53:53.499133 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499096    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499382 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499139    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499382 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499203    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499382 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499224    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-web-config\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499382 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499246    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-config\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499382 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499278    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499382 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499304    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499382 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499328    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zzpk\" (UniqueName: \"kubernetes.io/projected/4e319b56-b61c-4ca4-b609-bcd7a96258ff-kube-api-access-4zzpk\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499386    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499412    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499439    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499466    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499510    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e319b56-b61c-4ca4-b609-bcd7a96258ff-config-out\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499538    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499585    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e319b56-b61c-4ca4-b609-bcd7a96258ff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499610    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499638    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499663    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499695    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.499735 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.499725    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600552 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600463    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e319b56-b61c-4ca4-b609-bcd7a96258ff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600552 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600513    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600755 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600637    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600755 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600678    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600755 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600714    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600755 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600745    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600800    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600827    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600864    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600890    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-web-config\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.600945 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600916    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-config\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600959    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.600992    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601025    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zzpk\" (UniqueName: \"kubernetes.io/projected/4e319b56-b61c-4ca4-b609-bcd7a96258ff-kube-api-access-4zzpk\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601064    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601093    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601105    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601120    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601160    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601583 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601162    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601583 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601229    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e319b56-b61c-4ca4-b609-bcd7a96258ff-config-out\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601583 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601269    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601583 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601371    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601777 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601619    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.601777 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.601688    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.603547 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.603478    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e319b56-b61c-4ca4-b609-bcd7a96258ff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.603547 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.603529    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.603943 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.603912    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-config\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.603943 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.603914    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.604133 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.603968    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e319b56-b61c-4ca4-b609-bcd7a96258ff-config-out\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.604461 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.604360    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.604561 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.604486    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.605125 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.605103    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4e319b56-b61c-4ca4-b609-bcd7a96258ff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.605261 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.605113    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.606178 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.606146    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.606254 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.606232    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.606821 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.606797    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.606936 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.606920    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-web-config\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.607041 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.607025    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4e319b56-b61c-4ca4-b609-bcd7a96258ff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.610915 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.610897    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zzpk\" (UniqueName: \"kubernetes.io/projected/4e319b56-b61c-4ca4-b609-bcd7a96258ff-kube-api-access-4zzpk\") pod \"prometheus-k8s-0\" (UID: \"4e319b56-b61c-4ca4-b609-bcd7a96258ff\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.702314 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.702259    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:53:53.841287 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:53.841261    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 13:53:53.843041 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:53:53.843016    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e319b56_b61c_4ca4_b609_bcd7a96258ff.slice/crio-2e26f9ac3d1879950dd2f1791873d6509ce9c2265253a6735ea8a1feac7b5bbb WatchSource:0}: Error finding container 2e26f9ac3d1879950dd2f1791873d6509ce9c2265253a6735ea8a1feac7b5bbb: Status 404 returned error can't find the container with id 2e26f9ac3d1879950dd2f1791873d6509ce9c2265253a6735ea8a1feac7b5bbb
Jul 15 13:53:54.332138 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:54.332106    2573 generic.go:356] "Generic (PLEG): container finished" podID="4e319b56-b61c-4ca4-b609-bcd7a96258ff" containerID="1ced07a46c84c149ff2eedfa4f2b2414203cc8f277cb838e1a09dfb8689a31c1" exitCode=0
Jul 15 13:53:54.332565 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:54.332179    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e319b56-b61c-4ca4-b609-bcd7a96258ff","Type":"ContainerDied","Data":"1ced07a46c84c149ff2eedfa4f2b2414203cc8f277cb838e1a09dfb8689a31c1"}
Jul 15 13:53:54.332565 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:54.332232    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e319b56-b61c-4ca4-b609-bcd7a96258ff","Type":"ContainerStarted","Data":"2e26f9ac3d1879950dd2f1791873d6509ce9c2265253a6735ea8a1feac7b5bbb"}
Jul 15 13:53:55.339428 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:55.339389    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e319b56-b61c-4ca4-b609-bcd7a96258ff","Type":"ContainerStarted","Data":"a765b00905f900b96f9e15a1a55d2cc263aa8a6d0bd55ee31409e603e0dd30a2"}
Jul 15 13:53:55.339792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:55.339489    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e319b56-b61c-4ca4-b609-bcd7a96258ff","Type":"ContainerStarted","Data":"ff9f890b2204079a8f03ac67d46a0562ed5959ef479bbdefccea3f919562c2a9"}
Jul 15 13:53:55.339792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:55.339516    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e319b56-b61c-4ca4-b609-bcd7a96258ff","Type":"ContainerStarted","Data":"585cd4f4ed8911107bef27f9a799855424bd76f938a1555d40a5bbc8ed655d9b"}
Jul 15 13:53:55.339792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:55.339539    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e319b56-b61c-4ca4-b609-bcd7a96258ff","Type":"ContainerStarted","Data":"08d6b5401af1fc18619837060b4572f64ccbdf7b40b81b86b4e94ae81a48a86b"}
Jul 15 13:53:55.339792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:55.339571    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e319b56-b61c-4ca4-b609-bcd7a96258ff","Type":"ContainerStarted","Data":"0bd1841af62e857dabfd4d6896d494a1a648256daec56525859e67ba961d2b54"}
Jul 15 13:53:55.339792 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:55.339599    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e319b56-b61c-4ca4-b609-bcd7a96258ff","Type":"ContainerStarted","Data":"23df0410fcaf93e1c3fe3997f4bafa9424beaac4e9447c06d40452887374b33a"}
Jul 15 13:53:55.371370 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:55.371295    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.351899688 podStartE2EDuration="2.371269682s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.369994ms" isStatefulPod=false podCreationTimestamp="2026-07-15 13:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:53:55.370399814 +0000 UTC m=+156.490205394" watchObservedRunningTime="2026-07-15 13:53:55.371269682 +0000 UTC m=+156.491075264"
Jul 15 13:53:58.702949 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:53:58.702900    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:54:53.702737 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:54:53.702689    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:54:53.718293 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:54:53.718265    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:54:54.591588 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:54:54.591557    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 13:55:19.953776 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:19.953743    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-njz2w"]
Jul 15 13:55:19.958658 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:19.958617    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:19.960711 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:19.960679    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-425xb\" (UniqueName: \"kubernetes.io/projected/8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4-kube-api-access-425xb\") pod \"cert-manager-webhook-6486ffd997-njz2w\" (UID: \"8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:19.960845 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:19.960805    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-njz2w\" (UID: \"8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:19.961780 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:19.961758    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 15 13:55:19.962772 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:19.962756    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 15 13:55:19.962848 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:19.962754    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-szpff\""
Jul 15 13:55:19.968712 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:19.968689    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-njz2w"]
Jul 15 13:55:20.061627 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:20.061592    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-425xb\" (UniqueName: \"kubernetes.io/projected/8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4-kube-api-access-425xb\") pod \"cert-manager-webhook-6486ffd997-njz2w\" (UID: \"8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:20.061797 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:20.061733    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-njz2w\" (UID: \"8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:20.074678 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:20.074640    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-425xb\" (UniqueName: \"kubernetes.io/projected/8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4-kube-api-access-425xb\") pod \"cert-manager-webhook-6486ffd997-njz2w\" (UID: \"8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:20.076634 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:20.076611    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-njz2w\" (UID: \"8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:20.278584 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:20.278552    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:20.412980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:20.412956    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-njz2w"]
Jul 15 13:55:20.415724 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:55:20.415687    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a1f3ab9_83a0_43a6_9e4d_9e1f7ac2b6c4.slice/crio-5646ffec404646c1e84f232f228616d85a380ddb9a18185dfef81001a4639f12 WatchSource:0}: Error finding container 5646ffec404646c1e84f232f228616d85a380ddb9a18185dfef81001a4639f12: Status 404 returned error can't find the container with id 5646ffec404646c1e84f232f228616d85a380ddb9a18185dfef81001a4639f12
Jul 15 13:55:20.681193 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:20.681100    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w" event={"ID":"8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4","Type":"ContainerStarted","Data":"5646ffec404646c1e84f232f228616d85a380ddb9a18185dfef81001a4639f12"}
Jul 15 13:55:24.699237 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:24.699199    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w" event={"ID":"8a1f3ab9-83a0-43a6-9e4d-9e1f7ac2b6c4","Type":"ContainerStarted","Data":"87416de012594bf993540ce817858558b079f2e32af7e71d9b65d2f94d22f69c"}
Jul 15 13:55:24.699632 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:24.699259    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:24.719658 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:24.719607    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w" podStartSLOduration=2.047039067 podStartE2EDuration="5.719591002s" totalImagesPullingTime="3.672551935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:55:24.719019773 +0000 UTC m=+245.838825354" watchObservedRunningTime="2026-07-15 13:55:24.719591002 +0000 UTC m=+245.839396582"
Jul 15 13:55:29.793405 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.793365    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-74xpb"]
Jul 15 13:55:29.796774 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.796756    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-74xpb"
Jul 15 13:55:29.799606 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.799583    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-j69w8\""
Jul 15 13:55:29.809500 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.809471    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-74xpb"]
Jul 15 13:55:29.849277 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.849233    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78ead90c-6d72-40d7-8ef6-787a8f3d2196-bound-sa-token\") pod \"cert-manager-55b4d954d5-74xpb\" (UID: \"78ead90c-6d72-40d7-8ef6-787a8f3d2196\") " pod="cert-manager/cert-manager-55b4d954d5-74xpb"
Jul 15 13:55:29.849277 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.849274    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqc5t\" (UniqueName: \"kubernetes.io/projected/78ead90c-6d72-40d7-8ef6-787a8f3d2196-kube-api-access-fqc5t\") pod \"cert-manager-55b4d954d5-74xpb\" (UID: \"78ead90c-6d72-40d7-8ef6-787a8f3d2196\") " pod="cert-manager/cert-manager-55b4d954d5-74xpb"
Jul 15 13:55:29.950722 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.950673    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78ead90c-6d72-40d7-8ef6-787a8f3d2196-bound-sa-token\") pod \"cert-manager-55b4d954d5-74xpb\" (UID: \"78ead90c-6d72-40d7-8ef6-787a8f3d2196\") " pod="cert-manager/cert-manager-55b4d954d5-74xpb"
Jul 15 13:55:29.950722 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.950719    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqc5t\" (UniqueName: \"kubernetes.io/projected/78ead90c-6d72-40d7-8ef6-787a8f3d2196-kube-api-access-fqc5t\") pod \"cert-manager-55b4d954d5-74xpb\" (UID: \"78ead90c-6d72-40d7-8ef6-787a8f3d2196\") " pod="cert-manager/cert-manager-55b4d954d5-74xpb"
Jul 15 13:55:29.959787 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.959752    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqc5t\" (UniqueName: \"kubernetes.io/projected/78ead90c-6d72-40d7-8ef6-787a8f3d2196-kube-api-access-fqc5t\") pod \"cert-manager-55b4d954d5-74xpb\" (UID: \"78ead90c-6d72-40d7-8ef6-787a8f3d2196\") " pod="cert-manager/cert-manager-55b4d954d5-74xpb"
Jul 15 13:55:29.959935 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:29.959811    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78ead90c-6d72-40d7-8ef6-787a8f3d2196-bound-sa-token\") pod \"cert-manager-55b4d954d5-74xpb\" (UID: \"78ead90c-6d72-40d7-8ef6-787a8f3d2196\") " pod="cert-manager/cert-manager-55b4d954d5-74xpb"
Jul 15 13:55:30.105729 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:30.105630    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-74xpb"
Jul 15 13:55:30.239274 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:30.239240    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-74xpb"]
Jul 15 13:55:30.242161 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:55:30.242121    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78ead90c_6d72_40d7_8ef6_787a8f3d2196.slice/crio-0fc5bd799431db1397b8555106ce5fdd0b191f05f7439173fde1b6dacb67ce5e WatchSource:0}: Error finding container 0fc5bd799431db1397b8555106ce5fdd0b191f05f7439173fde1b6dacb67ce5e: Status 404 returned error can't find the container with id 0fc5bd799431db1397b8555106ce5fdd0b191f05f7439173fde1b6dacb67ce5e
Jul 15 13:55:30.706279 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:30.706249    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-njz2w"
Jul 15 13:55:30.724176 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:30.724146    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-74xpb" event={"ID":"78ead90c-6d72-40d7-8ef6-787a8f3d2196","Type":"ContainerStarted","Data":"40c846ea699c393e32b8699b9cf034c07007bc671ef30b5a88fa652e620a3d98"}
Jul 15 13:55:30.724370 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:30.724184    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-74xpb" event={"ID":"78ead90c-6d72-40d7-8ef6-787a8f3d2196","Type":"ContainerStarted","Data":"0fc5bd799431db1397b8555106ce5fdd0b191f05f7439173fde1b6dacb67ce5e"}
Jul 15 13:55:30.791646 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:55:30.791588    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-74xpb" podStartSLOduration=1.791538719 podStartE2EDuration="1.791538719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:55:30.791221765 +0000 UTC m=+251.911027344" watchObservedRunningTime="2026-07-15 13:55:30.791538719 +0000 UTC m=+251.911344300"
Jul 15 13:56:00.506788 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.506750    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"]
Jul 15 13:56:00.510914 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.510891    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.513852 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.513827    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 13:56:00.515096 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.515075    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 13:56:00.515173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.515100    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 13:56:00.515173 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.515086    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 13:56:00.515831 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.515815    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 13:56:00.515879 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.515831    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d4ksz\""
Jul 15 13:56:00.522757 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.522730    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g88s7\" (UniqueName: \"kubernetes.io/projected/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-kube-api-access-g88s7\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.522912 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.522816    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-cert\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.522912 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.522840    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-metrics-cert\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.522912 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.522874    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-manager-config\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.523845 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.523820    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"]
Jul 15 13:56:00.623605 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.623568    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-manager-config\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.623807 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.623629    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g88s7\" (UniqueName: \"kubernetes.io/projected/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-kube-api-access-g88s7\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.623807 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.623693    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-cert\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.623807 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.623716    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-metrics-cert\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.624447 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.624421    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-manager-config\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.626316 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.626294    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-metrics-cert\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.626421 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.626353    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-cert\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.634681 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.634657    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g88s7\" (UniqueName: \"kubernetes.io/projected/3c9543b1-2ae2-4b78-80d7-b90d9d68b802-kube-api-access-g88s7\") pod \"lws-controller-manager-6b767cd864-c575r\" (UID: \"3c9543b1-2ae2-4b78-80d7-b90d9d68b802\") " pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.820740 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.820694    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:00.959762 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:00.959731    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"]
Jul 15 13:56:00.962651 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:56:00.962623    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c9543b1_2ae2_4b78_80d7_b90d9d68b802.slice/crio-c6edaa984078ce9ec1bd3f1852c1c37bc46ead93b86d530a440f1c0251c3283d WatchSource:0}: Error finding container c6edaa984078ce9ec1bd3f1852c1c37bc46ead93b86d530a440f1c0251c3283d: Status 404 returned error can't find the container with id c6edaa984078ce9ec1bd3f1852c1c37bc46ead93b86d530a440f1c0251c3283d
Jul 15 13:56:01.852091 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:01.852050    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r" event={"ID":"3c9543b1-2ae2-4b78-80d7-b90d9d68b802","Type":"ContainerStarted","Data":"c6edaa984078ce9ec1bd3f1852c1c37bc46ead93b86d530a440f1c0251c3283d"}
Jul 15 13:56:03.862861 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:03.862828    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r" event={"ID":"3c9543b1-2ae2-4b78-80d7-b90d9d68b802","Type":"ContainerStarted","Data":"f1cd6d63c413354a9ace8923c0e82f1675d87842fe6ea77bc3dc1f4300ee7722"}
Jul 15 13:56:03.863274 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:03.862940    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:03.885883 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:03.885830    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r" podStartSLOduration=1.500549009 podStartE2EDuration="3.885814796s" totalImagesPullingTime="2.385265787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:03.884445773 +0000 UTC m=+285.004251353" watchObservedRunningTime="2026-07-15 13:56:03.885814796 +0000 UTC m=+285.005620376"
Jul 15 13:56:13.878420 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.878386    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"]
Jul 15 13:56:13.881980 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.881958    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:13.905720 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:13.905672    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" podUID="327f3e07-1ae0-49cc-8591-80bbc95622cf" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:13.905720 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:13.905697    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-fdjm8\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fdjm8\"" type="*v1.Secret"
Jul 15 13:56:13.905897 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:13.905714    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 15 13:56:13.905897 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:13.905714    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 15 13:56:13.905897 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:13.905767    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret"
Jul 15 13:56:13.905897 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.905775    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 13:56:13.935631 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.935600    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"]
Jul 15 13:56:13.939260 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.939239    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:13.939396 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.939274    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/327f3e07-1ae0-49cc-8591-80bbc95622cf-manifests\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:13.939396 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.939309    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:13.939488 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.939398    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/327f3e07-1ae0-49cc-8591-80bbc95622cf-tmp\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:13.939488 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:13.939436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psq82\" (UniqueName: \"kubernetes.io/projected/327f3e07-1ae0-49cc-8591-80bbc95622cf-kube-api-access-psq82\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:14.040612 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:14.040580    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psq82\" (UniqueName: \"kubernetes.io/projected/327f3e07-1ae0-49cc-8591-80bbc95622cf-kube-api-access-psq82\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:14.040782 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:14.040635    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:14.040782 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:14.040668    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/327f3e07-1ae0-49cc-8591-80bbc95622cf-manifests\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:14.040782 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:14.040698    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:14.040782 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:14.040751    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/327f3e07-1ae0-49cc-8591-80bbc95622cf-tmp\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:14.046883 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:14.041495    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/327f3e07-1ae0-49cc-8591-80bbc95622cf-manifests\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:14.046883 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:14.043732    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/327f3e07-1ae0-49cc-8591-80bbc95622cf-tmp\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:14.869397 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:14.869367    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6b767cd864-c575r"
Jul 15 13:56:15.041198 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:15.041159    2573 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition
Jul 15 13:56:15.041628 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:15.041241    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-apiservice-cert podName:327f3e07-1ae0-49cc-8591-80bbc95622cf nodeName:}" failed. No retries permitted until 2026-07-15 13:56:15.541223136 +0000 UTC m=+296.661028694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-apiservice-cert") pod "opendatahub-operator-controller-manager-58577d4d66-2l6zh" (UID: "327f3e07-1ae0-49cc-8591-80bbc95622cf") : failed to sync secret cache: timed out waiting for the condition
Jul 15 13:56:15.041628 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:15.041166    2573 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition
Jul 15 13:56:15.041628 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:15.041378    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-webhook-cert podName:327f3e07-1ae0-49cc-8591-80bbc95622cf nodeName:}" failed. No retries permitted until 2026-07-15 13:56:15.54136299 +0000 UTC m=+296.661168548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-webhook-cert") pod "opendatahub-operator-controller-manager-58577d4d66-2l6zh" (UID: "327f3e07-1ae0-49cc-8591-80bbc95622cf") : failed to sync secret cache: timed out waiting for the condition
Jul 15 13:56:15.073385 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:15.073357    2573 projected.go:291] Couldn't get configMap opendatahub/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 15 13:56:15.179263 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.179181    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 13:56:15.183697 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:15.183680    2573 projected.go:196] Error preparing data for projected volume kube-api-access-psq82 for pod opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh: failed to sync configmap cache: timed out waiting for the condition
Jul 15 13:56:15.183766 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:15.183755    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/327f3e07-1ae0-49cc-8591-80bbc95622cf-kube-api-access-psq82 podName:327f3e07-1ae0-49cc-8591-80bbc95622cf nodeName:}" failed. No retries permitted until 2026-07-15 13:56:15.683740089 +0000 UTC m=+296.803545648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-psq82" (UniqueName: "kubernetes.io/projected/327f3e07-1ae0-49cc-8591-80bbc95622cf-kube-api-access-psq82") pod "opendatahub-operator-controller-manager-58577d4d66-2l6zh" (UID: "327f3e07-1ae0-49cc-8591-80bbc95622cf") : failed to sync configmap cache: timed out waiting for the condition
Jul 15 13:56:15.294040 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.294004    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fdjm8\""
Jul 15 13:56:15.304050 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.304022    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 13:56:15.361565 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.361532    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 13:56:15.553426 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.553394    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:15.553602 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.553487    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:15.555823 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.555796    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:15.555823 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.555819    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/327f3e07-1ae0-49cc-8591-80bbc95622cf-webhook-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:15.754709 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.754666    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psq82\" (UniqueName: \"kubernetes.io/projected/327f3e07-1ae0-49cc-8591-80bbc95622cf-kube-api-access-psq82\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:15.757120 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.757098    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psq82\" (UniqueName: \"kubernetes.io/projected/327f3e07-1ae0-49cc-8591-80bbc95622cf-kube-api-access-psq82\") pod \"opendatahub-operator-controller-manager-58577d4d66-2l6zh\" (UID: \"327f3e07-1ae0-49cc-8591-80bbc95622cf\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:15.991789 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:15.991707    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:16.153150 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:16.153119    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"]
Jul 15 13:56:16.157538 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:56:16.157509    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327f3e07_1ae0_49cc_8591_80bbc95622cf.slice/crio-66a9cb1edbf2a9412050b92d28321276ba3f93864353b0c0192b295a745c8a2e WatchSource:0}: Error finding container 66a9cb1edbf2a9412050b92d28321276ba3f93864353b0c0192b295a745c8a2e: Status 404 returned error can't find the container with id 66a9cb1edbf2a9412050b92d28321276ba3f93864353b0c0192b295a745c8a2e
Jul 15 13:56:16.917184 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:16.917143    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh" event={"ID":"327f3e07-1ae0-49cc-8591-80bbc95622cf","Type":"ContainerStarted","Data":"66a9cb1edbf2a9412050b92d28321276ba3f93864353b0c0192b295a745c8a2e"}
Jul 15 13:56:19.264870 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:19.264790    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 13:56:19.268096 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:19.268054    2573 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 13:56:19.275915 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:19.275886    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 13:56:19.933253 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:19.933215    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh" event={"ID":"327f3e07-1ae0-49cc-8591-80bbc95622cf","Type":"ContainerStarted","Data":"578d3f2d2eaad503754939252749e02ea520e77a34d9562aa1cda1b4c4cf7449"}
Jul 15 13:56:20.025459 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:56:20.025424    2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327f3e07_1ae0_49cc_8591_80bbc95622cf.slice/crio-578d3f2d2eaad503754939252749e02ea520e77a34d9562aa1cda1b4c4cf7449.scope\": RecentStats: unable to find data in memory cache]"
Jul 15 13:56:20.945614 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:20.945566    2573 generic.go:356] "Generic (PLEG): container finished" podID="327f3e07-1ae0-49cc-8591-80bbc95622cf" containerID="578d3f2d2eaad503754939252749e02ea520e77a34d9562aa1cda1b4c4cf7449" exitCode=0
Jul 15 13:56:20.945996 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:20.945650    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh" event={"ID":"327f3e07-1ae0-49cc-8591-80bbc95622cf","Type":"ContainerDied","Data":"578d3f2d2eaad503754939252749e02ea520e77a34d9562aa1cda1b4c4cf7449"}
Jul 15 13:56:21.952068 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:21.952028    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh" event={"ID":"327f3e07-1ae0-49cc-8591-80bbc95622cf","Type":"ContainerStarted","Data":"458b1f98f0edc03c0c592732dcfaaa0b1d875910ddc44d7c84ff86ba3e76ed94"}
Jul 15 13:56:21.952504 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:21.952163    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:21.980020 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:21.979972    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh" podStartSLOduration=4.892768123 podStartE2EDuration="8.979955177s" totalImagesPullingTime="3.053730264s" totalInitContainerRuntime="1.03345679s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:21.978211208 +0000 UTC m=+303.098016790" watchObservedRunningTime="2026-07-15 13:56:21.979955177 +0000 UTC m=+303.099760758"
Jul 15 13:56:32.958250 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:32.958214    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-2l6zh"
Jul 15 13:56:35.527226 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.527185    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"]
Jul 15 13:56:35.666197 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.666163    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"]
Jul 15 13:56:35.666376 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.666269    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.671149 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.671123    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 13:56:35.671149 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.671149    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 13:56:35.671346 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.671185    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 13:56:35.673958 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.673936    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 13:56:35.676276 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.676255    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x4ngj\""
Jul 15 13:56:35.839583 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.839493    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb4d1ff8-c081-4d10-988c-4349d7a2becc-tmp\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.839727 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.839598    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb4d1ff8-c081-4d10-988c-4349d7a2becc-tls-certs\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.839727 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.839658    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rdtf\" (UniqueName: \"kubernetes.io/projected/cb4d1ff8-c081-4d10-988c-4349d7a2becc-kube-api-access-4rdtf\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.940473 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.940429    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rdtf\" (UniqueName: \"kubernetes.io/projected/cb4d1ff8-c081-4d10-988c-4349d7a2becc-kube-api-access-4rdtf\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.940685 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.940487    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb4d1ff8-c081-4d10-988c-4349d7a2becc-tmp\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.940685 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.940549    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb4d1ff8-c081-4d10-988c-4349d7a2becc-tls-certs\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.942752 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.942728    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb4d1ff8-c081-4d10-988c-4349d7a2becc-tmp\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.942858 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.942780    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb4d1ff8-c081-4d10-988c-4349d7a2becc-tls-certs\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.964420 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.964387    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rdtf\" (UniqueName: \"kubernetes.io/projected/cb4d1ff8-c081-4d10-988c-4349d7a2becc-kube-api-access-4rdtf\") pod \"kube-auth-proxy-5b5998d775-tz8rr\" (UID: \"cb4d1ff8-c081-4d10-988c-4349d7a2becc\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:35.976371 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:35.976318    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"
Jul 15 13:56:36.173798 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:36.173763    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr"]
Jul 15 13:56:36.180213 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:56:36.180173    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4d1ff8_c081_4d10_988c_4349d7a2becc.slice/crio-f42847db81d27310ba2fa735195af3bc4ca66f9e923349ffe40647f83f86cf79 WatchSource:0}: Error finding container f42847db81d27310ba2fa735195af3bc4ca66f9e923349ffe40647f83f86cf79: Status 404 returned error can't find the container with id f42847db81d27310ba2fa735195af3bc4ca66f9e923349ffe40647f83f86cf79
Jul 15 13:56:36.183415 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:36.183395    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 13:56:37.021034 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:37.020999    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr" event={"ID":"cb4d1ff8-c081-4d10-988c-4349d7a2becc","Type":"ContainerStarted","Data":"f42847db81d27310ba2fa735195af3bc4ca66f9e923349ffe40647f83f86cf79"}
Jul 15 13:56:39.038799 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:39.038759    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr" event={"ID":"cb4d1ff8-c081-4d10-988c-4349d7a2becc","Type":"ContainerStarted","Data":"d787c73e042838e5603b3bb0a0832be224f0156f1949691e28702f973f05b6e9"}
Jul 15 13:56:39.062819 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:56:39.062747    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b5998d775-tz8rr" podStartSLOduration=1.353387606 podStartE2EDuration="4.062713076s" totalImagesPullingTime="2.70932547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:39.062077699 +0000 UTC m=+320.181883280" watchObservedRunningTime="2026-07-15 13:56:39.062713076 +0000 UTC m=+320.182518657"
Jul 15 13:58:38.915747 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:38.915710    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"]
Jul 15 13:58:38.919143 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:38.919124    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:38.922058 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:38.922019    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 15 13:58:38.922058 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:38.922041    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 13:58:38.922058 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:38.922050    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s2lxs\""
Jul 15 13:58:38.922952 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:38.922928    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 15 13:58:38.923052 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:38.922933    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 13:58:38.930867 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:38.930827    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"]
Jul 15 13:58:39.047313 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.047272    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjwfj\" (UniqueName: \"kubernetes.io/projected/555ef84b-66b5-45c3-8140-5516e9f0e1d3-kube-api-access-kjwfj\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.047313 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.047316    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/555ef84b-66b5-45c3-8140-5516e9f0e1d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.047558 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.047412    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/555ef84b-66b5-45c3-8140-5516e9f0e1d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.148246 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.148211    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjwfj\" (UniqueName: \"kubernetes.io/projected/555ef84b-66b5-45c3-8140-5516e9f0e1d3-kube-api-access-kjwfj\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.148246 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.148251    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/555ef84b-66b5-45c3-8140-5516e9f0e1d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.148496 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.148277    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/555ef84b-66b5-45c3-8140-5516e9f0e1d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.148899 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.148875    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/555ef84b-66b5-45c3-8140-5516e9f0e1d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.150796 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.150770    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/555ef84b-66b5-45c3-8140-5516e9f0e1d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.170623 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.170538    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjwfj\" (UniqueName: \"kubernetes.io/projected/555ef84b-66b5-45c3-8140-5516e9f0e1d3-kube-api-access-kjwfj\") pod \"kuadrant-console-plugin-66b5774985-9wwxv\" (UID: \"555ef84b-66b5-45c3-8140-5516e9f0e1d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.228507 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.228464    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"
Jul 15 13:58:39.370496 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.370372    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv"]
Jul 15 13:58:39.373371 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:58:39.373311    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod555ef84b_66b5_45c3_8140_5516e9f0e1d3.slice/crio-0f466786a4afd64a2ab4f6688c3011d8225284be26ad4ad92028d73965749527 WatchSource:0}: Error finding container 0f466786a4afd64a2ab4f6688c3011d8225284be26ad4ad92028d73965749527: Status 404 returned error can't find the container with id 0f466786a4afd64a2ab4f6688c3011d8225284be26ad4ad92028d73965749527
Jul 15 13:58:39.584400 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:39.584364    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv" event={"ID":"555ef84b-66b5-45c3-8140-5516e9f0e1d3","Type":"ContainerStarted","Data":"0f466786a4afd64a2ab4f6688c3011d8225284be26ad4ad92028d73965749527"}
Jul 15 13:58:50.031003 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.030914    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"]
Jul 15 13:58:50.063122 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.063089    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"]
Jul 15 13:58:50.063302 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.063220    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.070235 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.070204    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nwrzz\""
Jul 15 13:58:50.149938 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.149896    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"]
Jul 15 13:58:50.150371 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:58:50.150325    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-vzb7r], unattached volumes=[], failed to process volumes=[extensions-socket-volume kube-api-access-vzb7r]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg" podUID="a350f6ab-d2b4-4edc-b255-791bbca82f73"
Jul 15 13:58:50.161016 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.160973    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzb7r\" (UniqueName: \"kubernetes.io/projected/a350f6ab-d2b4-4edc-b255-791bbca82f73-kube-api-access-vzb7r\") pod \"kuadrant-operator-controller-manager-86878c769f-p84zg\" (UID: \"a350f6ab-d2b4-4edc-b255-791bbca82f73\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.161190 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.161133    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a350f6ab-d2b4-4edc-b255-791bbca82f73-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p84zg\" (UID: \"a350f6ab-d2b4-4edc-b255-791bbca82f73\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.171464 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.171386    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"]
Jul 15 13:58:50.235532 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.235495    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"]
Jul 15 13:58:50.243029 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.242998    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"
Jul 15 13:58:50.245971 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.245943    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6zwft\""
Jul 15 13:58:50.254898 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.254862    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"]
Jul 15 13:58:50.262006 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.261829    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a350f6ab-d2b4-4edc-b255-791bbca82f73-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p84zg\" (UID: \"a350f6ab-d2b4-4edc-b255-791bbca82f73\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.262006 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.261894    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzb7r\" (UniqueName: \"kubernetes.io/projected/a350f6ab-d2b4-4edc-b255-791bbca82f73-kube-api-access-vzb7r\") pod \"kuadrant-operator-controller-manager-86878c769f-p84zg\" (UID: \"a350f6ab-d2b4-4edc-b255-791bbca82f73\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.262006 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.261932    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn8tr\" (UniqueName: \"kubernetes.io/projected/ecdefa7e-8999-4d37-9345-5c427874951b-kube-api-access-jn8tr\") pod \"limitador-operator-controller-manager-859d7b78bd-vts9v\" (UID: \"ecdefa7e-8999-4d37-9345-5c427874951b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"
Jul 15 13:58:50.262323 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.262209    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a350f6ab-d2b4-4edc-b255-791bbca82f73-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p84zg\" (UID: \"a350f6ab-d2b4-4edc-b255-791bbca82f73\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.264887 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:58:50.264856    2573 projected.go:196] Error preparing data for projected volume kube-api-access-vzb7r for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-129-208.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object
Jul 15 13:58:50.265058 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:58:50.264955    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a350f6ab-d2b4-4edc-b255-791bbca82f73-kube-api-access-vzb7r podName:a350f6ab-d2b4-4edc-b255-791bbca82f73 nodeName:}" failed. No retries permitted until 2026-07-15 13:58:50.764933868 +0000 UTC m=+451.884739426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vzb7r" (UniqueName: "kubernetes.io/projected/a350f6ab-d2b4-4edc-b255-791bbca82f73-kube-api-access-vzb7r") pod "kuadrant-operator-controller-manager-86878c769f-p84zg" (UID: "a350f6ab-d2b4-4edc-b255-791bbca82f73") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-129-208.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object
Jul 15 13:58:50.363083 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.362998    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn8tr\" (UniqueName: \"kubernetes.io/projected/ecdefa7e-8999-4d37-9345-5c427874951b-kube-api-access-jn8tr\") pod \"limitador-operator-controller-manager-859d7b78bd-vts9v\" (UID: \"ecdefa7e-8999-4d37-9345-5c427874951b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"
Jul 15 13:58:50.375392 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.375353    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn8tr\" (UniqueName: \"kubernetes.io/projected/ecdefa7e-8999-4d37-9345-5c427874951b-kube-api-access-jn8tr\") pod \"limitador-operator-controller-manager-859d7b78bd-vts9v\" (UID: \"ecdefa7e-8999-4d37-9345-5c427874951b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"
Jul 15 13:58:50.559105 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.559066    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"
Jul 15 13:58:50.644228 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.644143    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.647251 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:58:50.647214    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-p84zg\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" podUID="a350f6ab-d2b4-4edc-b255-791bbca82f73" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.656142 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.656104    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.658966 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:58:50.658933    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-p84zg\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" podUID="a350f6ab-d2b4-4edc-b255-791bbca82f73" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:50.665315 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.665282    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a350f6ab-d2b4-4edc-b255-791bbca82f73-extensions-socket-volume\") pod \"a350f6ab-d2b4-4edc-b255-791bbca82f73\" (UID: \"a350f6ab-d2b4-4edc-b255-791bbca82f73\") "
Jul 15 13:58:50.665520 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.665506    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzb7r\" (UniqueName: \"kubernetes.io/projected/a350f6ab-d2b4-4edc-b255-791bbca82f73-kube-api-access-vzb7r\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:58:50.665614 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.665585    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a350f6ab-d2b4-4edc-b255-791bbca82f73-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a350f6ab-d2b4-4edc-b255-791bbca82f73" (UID: "a350f6ab-d2b4-4edc-b255-791bbca82f73"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:58:50.766422 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:50.766381    2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a350f6ab-d2b4-4edc-b255-791bbca82f73-extensions-socket-volume\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 13:58:51.407052 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:51.407003    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a350f6ab-d2b4-4edc-b255-791bbca82f73" path="/var/lib/kubelet/pods/a350f6ab-d2b4-4edc-b255-791bbca82f73/volumes"
Jul 15 13:58:51.649438 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:58:51.649400    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:58:51.655475 ip-10-0-129-208 kubenswrapper[2573]: E0715 13:58:51.655438    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-p84zg\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" podUID="a350f6ab-d2b4-4edc-b255-791bbca82f73" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p84zg"
Jul 15 13:59:02.106522 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:59:02.106501    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"]
Jul 15 13:59:02.108764 ip-10-0-129-208 kubenswrapper[2573]: W0715 13:59:02.108729    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecdefa7e_8999_4d37_9345_5c427874951b.slice/crio-d5f1d8446ee77dbdf044c8168d0f1c6bc4d75db584f820077996235b144a5f60 WatchSource:0}: Error finding container d5f1d8446ee77dbdf044c8168d0f1c6bc4d75db584f820077996235b144a5f60: Status 404 returned error can't find the container with id d5f1d8446ee77dbdf044c8168d0f1c6bc4d75db584f820077996235b144a5f60
Jul 15 13:59:02.709590 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:59:02.709500    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv" event={"ID":"555ef84b-66b5-45c3-8140-5516e9f0e1d3","Type":"ContainerStarted","Data":"b63a2dd36464f54707332a84a8197680b412c5859d66fb086f4c6898a7ce59a9"}
Jul 15 13:59:02.710910 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:59:02.710881    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v" event={"ID":"ecdefa7e-8999-4d37-9345-5c427874951b","Type":"ContainerStarted","Data":"d5f1d8446ee77dbdf044c8168d0f1c6bc4d75db584f820077996235b144a5f60"}
Jul 15 13:59:06.734538 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:59:06.734496    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v" event={"ID":"ecdefa7e-8999-4d37-9345-5c427874951b","Type":"ContainerStarted","Data":"5831a8ffba1f055fc12c756d740f6d09354da8e4f4848ba01bda62911183276d"}
Jul 15 13:59:06.735046 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:59:06.734604    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"
Jul 15 13:59:06.758576 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:59:06.758515    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9wwxv" podStartSLOduration=5.732475024 podStartE2EDuration="28.758493103s" totalImagesPullingTime="23.026018079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:02.768121944 +0000 UTC m=+463.887927525" watchObservedRunningTime="2026-07-15 13:59:06.758493103 +0000 UTC m=+467.878298685"
Jul 15 13:59:06.764355 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:59:06.760626    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v" podStartSLOduration=13.064984064 podStartE2EDuration="16.760598863s" totalImagesPullingTime="3.695614799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:06.759063833 +0000 UTC m=+467.878869414" watchObservedRunningTime="2026-07-15 13:59:06.760598863 +0000 UTC m=+467.880404444"
Jul 15 13:59:17.741105 ip-10-0-129-208 kubenswrapper[2573]: I0715 13:59:17.741072    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vts9v"
Jul 15 14:01:13.202306 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.202267    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"]
Jul 15 14:01:13.205736 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.205716    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:13.208617 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.208585    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8m8gn\""
Jul 15 14:01:13.208735 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.208651    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 14:01:13.216203 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.216171    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"]
Jul 15 14:01:13.286646 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.286603    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdw85\" (UniqueName: \"kubernetes.io/projected/edea8eea-8d49-410f-983a-8d17569c4c3b-kube-api-access-gdw85\") pod \"payload-pre-processing-74584bb9f8-ndj5q\" (UID: \"edea8eea-8d49-410f-983a-8d17569c4c3b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:13.286832 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.286704    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edea8eea-8d49-410f-983a-8d17569c4c3b-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-ndj5q\" (UID: \"edea8eea-8d49-410f-983a-8d17569c4c3b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:13.387513 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.387470    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edea8eea-8d49-410f-983a-8d17569c4c3b-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-ndj5q\" (UID: \"edea8eea-8d49-410f-983a-8d17569c4c3b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:13.387696 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.387536    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdw85\" (UniqueName: \"kubernetes.io/projected/edea8eea-8d49-410f-983a-8d17569c4c3b-kube-api-access-gdw85\") pod \"payload-pre-processing-74584bb9f8-ndj5q\" (UID: \"edea8eea-8d49-410f-983a-8d17569c4c3b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:13.388222 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.388200    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edea8eea-8d49-410f-983a-8d17569c4c3b-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-ndj5q\" (UID: \"edea8eea-8d49-410f-983a-8d17569c4c3b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:13.396540 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.396504    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdw85\" (UniqueName: \"kubernetes.io/projected/edea8eea-8d49-410f-983a-8d17569c4c3b-kube-api-access-gdw85\") pod \"payload-pre-processing-74584bb9f8-ndj5q\" (UID: \"edea8eea-8d49-410f-983a-8d17569c4c3b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:13.517158 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.517062    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:13.671595 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:13.671552    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"]
Jul 15 14:01:13.674863 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:01:13.674834    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedea8eea_8d49_410f_983a_8d17569c4c3b.slice/crio-afc441c52e5c6cf13f688f416b2d1b9c5e0ed5ff9c8f14f742169e765cced3d5 WatchSource:0}: Error finding container afc441c52e5c6cf13f688f416b2d1b9c5e0ed5ff9c8f14f742169e765cced3d5: Status 404 returned error can't find the container with id afc441c52e5c6cf13f688f416b2d1b9c5e0ed5ff9c8f14f742169e765cced3d5
Jul 15 14:01:14.318471 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:14.318432    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q" event={"ID":"edea8eea-8d49-410f-983a-8d17569c4c3b","Type":"ContainerStarted","Data":"afc441c52e5c6cf13f688f416b2d1b9c5e0ed5ff9c8f14f742169e765cced3d5"}
Jul 15 14:01:15.324938 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:15.324901    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q" event={"ID":"edea8eea-8d49-410f-983a-8d17569c4c3b","Type":"ContainerStarted","Data":"cf29147a817c1962df2ed3d78a21a6c7f16cc289f735ef7f70f6a4d7aa38805c"}
Jul 15 14:01:15.325327 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:15.324977    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:19.306803 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:19.306775    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:01:19.310238 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:19.310213    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:01:26.332253 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:26.332223    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q"
Jul 15 14:01:26.351689 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:26.351638    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ndj5q" podStartSLOduration=11.914419376 podStartE2EDuration="13.351622422s" totalImagesPullingTime="1.437203046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:01:15.359770351 +0000 UTC m=+596.479575932" watchObservedRunningTime="2026-07-15 14:01:26.351622422 +0000 UTC m=+607.471428002"
Jul 15 14:01:44.793137 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.793039    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"]
Jul 15 14:01:44.796866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.796838    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.799702 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.799673    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 14:01:44.800786 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.800755    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 15 14:01:44.800786 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.800758    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 14:01:44.800997 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.800854    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-czstb\""
Jul 15 14:01:44.806323 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.806297    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"]
Jul 15 14:01:44.881668 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.881625    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.881857 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.881686    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.881857 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.881710    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm69c\" (UniqueName: \"kubernetes.io/projected/eb838ea1-1b23-4696-adbd-bd72a7135e05-kube-api-access-gm69c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.881857 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.881755    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb838ea1-1b23-4696-adbd-bd72a7135e05-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.881857 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.881821    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.881992 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.881864    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.982733 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.982692    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.982733 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.982736    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm69c\" (UniqueName: \"kubernetes.io/projected/eb838ea1-1b23-4696-adbd-bd72a7135e05-kube-api-access-gm69c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.982956 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.982786    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb838ea1-1b23-4696-adbd-bd72a7135e05-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.982956 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.982814    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.982956 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.982834    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.982956 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.982863    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.983204 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.983180    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.983267 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.983223    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.983347 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.983321    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.985151 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.985130    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb838ea1-1b23-4696-adbd-bd72a7135e05-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.985430 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.985411    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb838ea1-1b23-4696-adbd-bd72a7135e05-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:44.991581 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:44.991553    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm69c\" (UniqueName: \"kubernetes.io/projected/eb838ea1-1b23-4696-adbd-bd72a7135e05-kube-api-access-gm69c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tzh8d\" (UID: \"eb838ea1-1b23-4696-adbd-bd72a7135e05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:45.109052 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:45.108954    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:45.250788 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:45.250757    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"]
Jul 15 14:01:45.252547 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:01:45.252514    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb838ea1_1b23_4696_adbd_bd72a7135e05.slice/crio-afda526f38b15e8bd8a0c05bc096abd3dc451957cc513bc7510137c85935e26f WatchSource:0}: Error finding container afda526f38b15e8bd8a0c05bc096abd3dc451957cc513bc7510137c85935e26f: Status 404 returned error can't find the container with id afda526f38b15e8bd8a0c05bc096abd3dc451957cc513bc7510137c85935e26f
Jul 15 14:01:45.253969 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:45.253947    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 14:01:45.468726 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:45.468633    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d" event={"ID":"eb838ea1-1b23-4696-adbd-bd72a7135e05","Type":"ContainerStarted","Data":"afda526f38b15e8bd8a0c05bc096abd3dc451957cc513bc7510137c85935e26f"}
Jul 15 14:01:47.481573 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:47.481532    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d" event={"ID":"eb838ea1-1b23-4696-adbd-bd72a7135e05","Type":"ContainerStarted","Data":"f9a9111af2ad5edf7fc796d3b0b2846910a12347a58093f83be1a144546ea62c"}
Jul 15 14:01:47.502399 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:47.502344    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d" podStartSLOduration=2.143756186 podStartE2EDuration="3.502314534s" totalImagesPullingTime="1.358558348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:01:47.500385539 +0000 UTC m=+628.620191121" watchObservedRunningTime="2026-07-15 14:01:47.502314534 +0000 UTC m=+628.622120114"
Jul 15 14:01:55.109730 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:55.109687    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:55.110238 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:55.109839    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:55.122685 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:55.122649    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:01:55.533119 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:01:55.533084    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tzh8d"
Jul 15 14:02:34.111953 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.111918    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"]
Jul 15 14:02:34.115765 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.115740    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.119134 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.119108    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 15 14:02:34.125287 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.125262    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"]
Jul 15 14:02:34.244737 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.244689    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.244922 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.244762    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtp4b\" (UniqueName: \"kubernetes.io/projected/40f5cfb9-f05a-47af-8d60-8795a068fd23-kube-api-access-gtp4b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.244922 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.244788    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f5cfb9-f05a-47af-8d60-8795a068fd23-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.244922 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.244822    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.244922 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.244895    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.245059 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.244947    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.345619 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.345573    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f5cfb9-f05a-47af-8d60-8795a068fd23-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.345819 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.345641    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.345819 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.345672    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.345819 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.345706    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.345819 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.345742    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.345819 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.345813    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtp4b\" (UniqueName: \"kubernetes.io/projected/40f5cfb9-f05a-47af-8d60-8795a068fd23-kube-api-access-gtp4b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.346180 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.346134    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.346180 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.346174    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.346366 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.346208    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.348020 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.348002    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f5cfb9-f05a-47af-8d60-8795a068fd23-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.348196 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.348179    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f5cfb9-f05a-47af-8d60-8795a068fd23-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.364833 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.364754    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtp4b\" (UniqueName: \"kubernetes.io/projected/40f5cfb9-f05a-47af-8d60-8795a068fd23-kube-api-access-gtp4b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh\" (UID: \"40f5cfb9-f05a-47af-8d60-8795a068fd23\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.426100 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.426052    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:34.571905 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.571880    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"]
Jul 15 14:02:34.574499 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:02:34.574466    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40f5cfb9_f05a_47af_8d60_8795a068fd23.slice/crio-53fe1ea9ebf8d8aab8d011a7a3216661d9439da8d7880da6a21f63adcf09e651 WatchSource:0}: Error finding container 53fe1ea9ebf8d8aab8d011a7a3216661d9439da8d7880da6a21f63adcf09e651: Status 404 returned error can't find the container with id 53fe1ea9ebf8d8aab8d011a7a3216661d9439da8d7880da6a21f63adcf09e651
Jul 15 14:02:34.707703 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:34.707616    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh" event={"ID":"40f5cfb9-f05a-47af-8d60-8795a068fd23","Type":"ContainerStarted","Data":"53fe1ea9ebf8d8aab8d011a7a3216661d9439da8d7880da6a21f63adcf09e651"}
Jul 15 14:02:35.714560 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:35.714523    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh" event={"ID":"40f5cfb9-f05a-47af-8d60-8795a068fd23","Type":"ContainerStarted","Data":"053517f222312ac87bd9237fa3c6ce79aa81790dfeb134b94c19da6c145328ab"}
Jul 15 14:02:35.734155 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:35.734099    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh" podStartSLOduration=1.445242725 podStartE2EDuration="1.734072215s" totalImagesPullingTime="288.82949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:02:35.732715793 +0000 UTC m=+676.852521376" watchObservedRunningTime="2026-07-15 14:02:35.734072215 +0000 UTC m=+676.853877795"
Jul 15 14:02:44.426476 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:44.426437    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:44.426476 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:44.426487    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:44.439301 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:44.439271    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:02:44.768796 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:02:44.768703    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh"
Jul 15 14:06:19.347788 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:06:19.347758    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:06:19.348949 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:06:19.348931    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:11:19.388916 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:11:19.388886    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:11:19.389456 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:11:19.389083    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:16:19.429186 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:16:19.429142    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:16:19.430124 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:16:19.430100    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:17:00.447742 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.447701    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"]
Jul 15 14:17:00.451142 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.451121    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:00.454278 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.454247    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-af5d844d-serving-cert\""
Jul 15 14:17:00.454853 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.454823    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:17:00.455501 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.455477    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:17:00.455846 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.455829    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:17:00.476781 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.476737    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"]
Jul 15 14:17:00.548777 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.548736    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-openshift-service-ca\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:00.548777 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.548790    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hlq8\" (UniqueName: \"kubernetes.io/projected/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-kube-api-access-4hlq8\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:00.548998 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.548881    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:00.649732 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.649684    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-openshift-service-ca\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:00.649732 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.649741    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hlq8\" (UniqueName: \"kubernetes.io/projected/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-kube-api-access-4hlq8\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:00.649950 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.649856    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:00.650006 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:17:00.649987    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-serving-cert: secret "maas-api-e2e-ait-af5d844d-serving-cert" not found
Jul 15 14:17:00.650073 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:17:00.650063    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls podName:cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1 nodeName:}" failed. No retries permitted until 2026-07-15 14:17:01.150046009 +0000 UTC m=+1542.269851569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls") pod "maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n" (UID: "cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1") : secret "maas-api-e2e-ait-af5d844d-serving-cert" not found
Jul 15 14:17:00.650464 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.650447    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-openshift-service-ca\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:00.662909 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:00.662882    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hlq8\" (UniqueName: \"kubernetes.io/projected/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-kube-api-access-4hlq8\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:01.155132 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:01.155087    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:01.157661 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:01.157632    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls\") pod \"maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:01.362448 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:01.362404    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:01.504166 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:01.504142    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"]
Jul 15 14:17:01.505830 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:17:01.505797    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc8ddac_7ec0_4a66_bd86_6f8b4bb61de1.slice/crio-5c90da23645112fe5b9a6c4e8d8356f9a247bdb9cfccb87b7a1f70f4ebb6fb66 WatchSource:0}: Error finding container 5c90da23645112fe5b9a6c4e8d8356f9a247bdb9cfccb87b7a1f70f4ebb6fb66: Status 404 returned error can't find the container with id 5c90da23645112fe5b9a6c4e8d8356f9a247bdb9cfccb87b7a1f70f4ebb6fb66
Jul 15 14:17:01.507129 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:01.507112    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 14:17:02.039739 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:02.039704    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n" event={"ID":"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1","Type":"ContainerStarted","Data":"5c90da23645112fe5b9a6c4e8d8356f9a247bdb9cfccb87b7a1f70f4ebb6fb66"}
Jul 15 14:17:05.058945 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.058908    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n" event={"ID":"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1","Type":"ContainerStarted","Data":"11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15"}
Jul 15 14:17:05.059407 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.059006    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:05.084704 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.084648    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n" podStartSLOduration=2.171871329 podStartE2EDuration="5.084631122s" totalImagesPullingTime="2.912759793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:17:05.082466359 +0000 UTC m=+1546.202271951" watchObservedRunningTime="2026-07-15 14:17:05.084631122 +0000 UTC m=+1546.204436703"
Jul 15 14:17:05.391321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.391231    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"]
Jul 15 14:17:05.394902 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.394877    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"
Jul 15 14:17:05.398658 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.398635    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\""
Jul 15 14:17:05.409118 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.409088    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"]
Jul 15 14:17:05.500308 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.500249    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn\" (UID: \"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"
Jul 15 14:17:05.601195 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.601153    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn\" (UID: \"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"
Jul 15 14:17:05.601843 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.601822    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn\" (UID: \"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"
Jul 15 14:17:05.707612 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.707524    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"
Jul 15 14:17:05.849389 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:05.849355    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"]
Jul 15 14:17:05.851397 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:17:05.851321    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ceb32f_0c7e_4b5a_8b77_02bc1d0301e8.slice/crio-c0ad2ef3d7568ed1e708877ac3da5f8d73ae2e7ea71e18dd7d82ca6e5b13e65b WatchSource:0}: Error finding container c0ad2ef3d7568ed1e708877ac3da5f8d73ae2e7ea71e18dd7d82ca6e5b13e65b: Status 404 returned error can't find the container with id c0ad2ef3d7568ed1e708877ac3da5f8d73ae2e7ea71e18dd7d82ca6e5b13e65b
Jul 15 14:17:06.066542 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:06.066497    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn" event={"ID":"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8","Type":"ContainerStarted","Data":"c0ad2ef3d7568ed1e708877ac3da5f8d73ae2e7ea71e18dd7d82ca6e5b13e65b"}
Jul 15 14:17:11.072143 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:11.072111    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:11.097187 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:11.097145    2573 generic.go:356] "Generic (PLEG): container finished" podID="95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" containerID="4513c91bb2e2211f2306007f474aa3b87d36fba0574b7692fa655d79e67f28c6" exitCode=7
Jul 15 14:17:11.097570 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:11.097544    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn" event={"ID":"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8","Type":"ContainerDied","Data":"4513c91bb2e2211f2306007f474aa3b87d36fba0574b7692fa655d79e67f28c6"}
Jul 15 14:17:11.098130 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:11.098092    2573 scope.go:117] "RemoveContainer" containerID="4513c91bb2e2211f2306007f474aa3b87d36fba0574b7692fa655d79e67f28c6"
Jul 15 14:17:12.104417 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:12.104293    2573 generic.go:356] "Generic (PLEG): container finished" podID="95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" containerID="c082461ac0421a919af8cd097d4b4f3387ea02279e7b361454aa8f52ca5c94e1" exitCode=0
Jul 15 14:17:12.104417 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:12.104385    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn" event={"ID":"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8","Type":"ContainerDied","Data":"c082461ac0421a919af8cd097d4b4f3387ea02279e7b361454aa8f52ca5c94e1"}
Jul 15 14:17:12.104851 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:12.104460    2573 scope.go:117] "RemoveContainer" containerID="4513c91bb2e2211f2306007f474aa3b87d36fba0574b7692fa655d79e67f28c6"
Jul 15 14:17:13.250460 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:13.250429    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"
Jul 15 14:17:13.383284 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:13.383244    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8-maas-api-service-ca\") pod \"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8\" (UID: \"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8\") "
Jul 15 14:17:13.383660 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:13.383636    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" (UID: "95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:17:13.484762 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:13.484713    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8-maas-api-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:17:14.116937 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:14.116897    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn" event={"ID":"95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8","Type":"ContainerDied","Data":"c0ad2ef3d7568ed1e708877ac3da5f8d73ae2e7ea71e18dd7d82ca6e5b13e65b"}
Jul 15 14:17:14.116937 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:14.116923    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn"
Jul 15 14:17:14.117162 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:14.116950    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0ad2ef3d7568ed1e708877ac3da5f8d73ae2e7ea71e18dd7d82ca6e5b13e65b"
Jul 15 14:17:16.092766 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.092727    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"]
Jul 15 14:17:16.093207 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.092961    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n" podUID="cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" containerName="maas-api" containerID="cri-o://11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15" gracePeriod=30
Jul 15 14:17:16.349977 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.349910    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:16.512714 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.512675    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-openshift-service-ca\") pod \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") "
Jul 15 14:17:16.512714 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.512720    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls\") pod \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") "
Jul 15 14:17:16.512937 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.512752    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hlq8\" (UniqueName: \"kubernetes.io/projected/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-kube-api-access-4hlq8\") pod \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\" (UID: \"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1\") "
Jul 15 14:17:16.513162 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.513116    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" (UID: "cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:17:16.515102 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.515065    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-kube-api-access-4hlq8" (OuterVolumeSpecName: "kube-api-access-4hlq8") pod "cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" (UID: "cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1"). InnerVolumeSpecName "kube-api-access-4hlq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:17:16.515102 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.515094    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" (UID: "cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:17:16.614314 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.614222    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:17:16.614314 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.614257    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:17:16.614314 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:16.614267    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hlq8\" (UniqueName: \"kubernetes.io/projected/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1-kube-api-access-4hlq8\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:17:17.132703 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.132663    2573 generic.go:356] "Generic (PLEG): container finished" podID="cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" containerID="11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15" exitCode=0
Jul 15 14:17:17.133107 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.132717    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"
Jul 15 14:17:17.133107 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.132757    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n" event={"ID":"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1","Type":"ContainerDied","Data":"11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15"}
Jul 15 14:17:17.133107 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.132817    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n" event={"ID":"cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1","Type":"ContainerDied","Data":"5c90da23645112fe5b9a6c4e8d8356f9a247bdb9cfccb87b7a1f70f4ebb6fb66"}
Jul 15 14:17:17.133107 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.132839    2573 scope.go:117] "RemoveContainer" containerID="11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15"
Jul 15 14:17:17.142427 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.142407    2573 scope.go:117] "RemoveContainer" containerID="11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15"
Jul 15 14:17:17.142762 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:17:17.142737    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15\": container with ID starting with 11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15 not found: ID does not exist" containerID="11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15"
Jul 15 14:17:17.142809 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.142776    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15"} err="failed to get container status \"11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15\": rpc error: code = NotFound desc = could not find container \"11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15\": container with ID starting with 11a750f91c6af146e1d626194279177526e13851c070fc8f3dff179f4cfd1e15 not found: ID does not exist"
Jul 15 14:17:17.165413 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.165350    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"]
Jul 15 14:17:17.170849 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.170814    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5d844d-c5b54f54f-qcb5n"]
Jul 15 14:17:17.406744 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:17.406660    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" path="/var/lib/kubelet/pods/cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1/volumes"
Jul 15 14:17:33.014703 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.014667    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"]
Jul 15 14:17:33.015149 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015074    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" containerName="maas-api"
Jul 15 14:17:33.015149 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015086    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" containerName="maas-api"
Jul 15 14:17:33.015149 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015094    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" containerName="revoke-keys"
Jul 15 14:17:33.015149 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015100    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" containerName="revoke-keys"
Jul 15 14:17:33.015149 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015112    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" containerName="revoke-keys"
Jul 15 14:17:33.015149 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015117    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" containerName="revoke-keys"
Jul 15 14:17:33.015351 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015190    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfc8ddac-7ec0-4a66-bd86-6f8b4bb61de1" containerName="maas-api"
Jul 15 14:17:33.015351 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015200    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" containerName="revoke-keys"
Jul 15 14:17:33.015351 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.015207    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8" containerName="revoke-keys"
Jul 15 14:17:33.024210 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.024176    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.027254 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.027230    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-30d2f55b-serving-cert\""
Jul 15 14:17:33.027691 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.027668    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:17:33.028447 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.028420    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:17:33.028732 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.028709    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:17:33.031724 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.031698    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"]
Jul 15 14:17:33.073219 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.073177    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-openshift-service-ca\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.073447 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.073267    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5b7c\" (UniqueName: \"kubernetes.io/projected/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-kube-api-access-b5b7c\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.073447 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.073319    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.174371 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.174296    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-openshift-service-ca\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.174565 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.174380    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5b7c\" (UniqueName: \"kubernetes.io/projected/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-kube-api-access-b5b7c\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.174565 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.174417    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.174680 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:17:33.174563    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-serving-cert: secret "maas-api-e2e-migrate-30d2f55b-serving-cert" not found
Jul 15 14:17:33.174680 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:17:33.174646    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls podName:83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b nodeName:}" failed. No retries permitted until 2026-07-15 14:17:33.674628668 +0000 UTC m=+1574.794434226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls") pod "maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57" (UID: "83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b") : secret "maas-api-e2e-migrate-30d2f55b-serving-cert" not found
Jul 15 14:17:33.175059 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.175037    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-openshift-service-ca\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.199971 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.199934    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5b7c\" (UniqueName: \"kubernetes.io/projected/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-kube-api-access-b5b7c\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.680736 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.680679    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.683190 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.683165    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls\") pod \"maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:33.937467 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:33.937375    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:34.095583 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:34.095552    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"]
Jul 15 14:17:34.097709 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:17:34.097680    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83deffe9_b7c4_4e9b_bc3c_e79fb7bea12b.slice/crio-9d0e7e5158ea615747275fd5159626adbe6147665c271041743420b361df1e7f WatchSource:0}: Error finding container 9d0e7e5158ea615747275fd5159626adbe6147665c271041743420b361df1e7f: Status 404 returned error can't find the container with id 9d0e7e5158ea615747275fd5159626adbe6147665c271041743420b361df1e7f
Jul 15 14:17:34.223604 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:34.223516    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57" event={"ID":"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b","Type":"ContainerStarted","Data":"9d0e7e5158ea615747275fd5159626adbe6147665c271041743420b361df1e7f"}
Jul 15 14:17:35.230101 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:35.230061    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57" event={"ID":"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b","Type":"ContainerStarted","Data":"7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86"}
Jul 15 14:17:35.230573 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:35.230161    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:17:35.252731 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:35.252682    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57" podStartSLOduration=2.7505144379999997 podStartE2EDuration="3.252662744s" totalImagesPullingTime="502.148306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:17:35.252592056 +0000 UTC m=+1576.372397644" watchObservedRunningTime="2026-07-15 14:17:35.252662744 +0000 UTC m=+1576.372468324"
Jul 15 14:17:41.239966 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:17:41.239934    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:18:27.365678 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.365590    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"]
Jul 15 14:18:27.369176 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.369153    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"
Jul 15 14:18:27.372391 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.372371    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\""
Jul 15 14:18:27.380856 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.380819    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"]
Jul 15 14:18:27.397811 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.397768    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f41902-068a-457f-a401-eb7bb64ec931-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc\" (UID: \"90f41902-068a-457f-a401-eb7bb64ec931\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"
Jul 15 14:18:27.498880 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.498842    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f41902-068a-457f-a401-eb7bb64ec931-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc\" (UID: \"90f41902-068a-457f-a401-eb7bb64ec931\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"
Jul 15 14:18:27.499547 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.499519    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f41902-068a-457f-a401-eb7bb64ec931-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc\" (UID: \"90f41902-068a-457f-a401-eb7bb64ec931\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"
Jul 15 14:18:27.680381 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.680264    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"
Jul 15 14:18:27.822711 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:27.822682    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"]
Jul 15 14:18:27.825220 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:18:27.825193    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f41902_068a_457f_a401_eb7bb64ec931.slice/crio-e8ea146030eaa6e55de485c6f3bb6984fd73f50cdbb05215a2e278dda0de163b WatchSource:0}: Error finding container e8ea146030eaa6e55de485c6f3bb6984fd73f50cdbb05215a2e278dda0de163b: Status 404 returned error can't find the container with id e8ea146030eaa6e55de485c6f3bb6984fd73f50cdbb05215a2e278dda0de163b
Jul 15 14:18:28.491486 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:28.491438    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc" event={"ID":"90f41902-068a-457f-a401-eb7bb64ec931","Type":"ContainerStarted","Data":"febf0f26924ec14d7e2940c0475901155da0ad608b413a4efe8ce881c5baf308"}
Jul 15 14:18:28.491889 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:28.491503    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc" event={"ID":"90f41902-068a-457f-a401-eb7bb64ec931","Type":"ContainerStarted","Data":"e8ea146030eaa6e55de485c6f3bb6984fd73f50cdbb05215a2e278dda0de163b"}
Jul 15 14:18:28.516976 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:28.516919    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc" podStartSLOduration=1.5169033330000001 podStartE2EDuration="1.516903333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:18:28.514697033 +0000 UTC m=+1629.634502613" watchObservedRunningTime="2026-07-15 14:18:28.516903333 +0000 UTC m=+1629.636708914"
Jul 15 14:18:29.498151 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:29.498117    2573 generic.go:356] "Generic (PLEG): container finished" podID="90f41902-068a-457f-a401-eb7bb64ec931" containerID="febf0f26924ec14d7e2940c0475901155da0ad608b413a4efe8ce881c5baf308" exitCode=0
Jul 15 14:18:29.498537 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:29.498205    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc" event={"ID":"90f41902-068a-457f-a401-eb7bb64ec931","Type":"ContainerDied","Data":"febf0f26924ec14d7e2940c0475901155da0ad608b413a4efe8ce881c5baf308"}
Jul 15 14:18:30.642663 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:30.642632    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"
Jul 15 14:18:30.729757 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:30.729723    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f41902-068a-457f-a401-eb7bb64ec931-maas-api-service-ca\") pod \"90f41902-068a-457f-a401-eb7bb64ec931\" (UID: \"90f41902-068a-457f-a401-eb7bb64ec931\") "
Jul 15 14:18:30.730124 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:30.730098    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90f41902-068a-457f-a401-eb7bb64ec931-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90f41902-068a-457f-a401-eb7bb64ec931" (UID: "90f41902-068a-457f-a401-eb7bb64ec931"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:18:30.830712 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:30.830675    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f41902-068a-457f-a401-eb7bb64ec931-maas-api-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:18:31.510504 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:31.510470    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc"
Jul 15 14:18:31.510686 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:31.510469    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc" event={"ID":"90f41902-068a-457f-a401-eb7bb64ec931","Type":"ContainerDied","Data":"e8ea146030eaa6e55de485c6f3bb6984fd73f50cdbb05215a2e278dda0de163b"}
Jul 15 14:18:31.510686 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:31.510594    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8ea146030eaa6e55de485c6f3bb6984fd73f50cdbb05215a2e278dda0de163b"
Jul 15 14:18:38.045187 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.045149    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"]
Jul 15 14:18:38.045659 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.045393    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57" podUID="83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" containerName="maas-api" containerID="cri-o://7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86" gracePeriod=30
Jul 15 14:18:38.291622 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.291594    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:18:38.401760 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.401672    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5b7c\" (UniqueName: \"kubernetes.io/projected/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-kube-api-access-b5b7c\") pod \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") "
Jul 15 14:18:38.401928 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.401766    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls\") pod \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") "
Jul 15 14:18:38.401928 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.401865    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-openshift-service-ca\") pod \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\" (UID: \"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b\") "
Jul 15 14:18:38.402244 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.402200    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" (UID: "83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:18:38.403789 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.403768    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" (UID: "83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:18:38.403874 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.403847    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-kube-api-access-b5b7c" (OuterVolumeSpecName: "kube-api-access-b5b7c") pod "83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" (UID: "83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b"). InnerVolumeSpecName "kube-api-access-b5b7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:18:38.503202 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.503161    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:18:38.503202 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.503196    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5b7c\" (UniqueName: \"kubernetes.io/projected/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-kube-api-access-b5b7c\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:18:38.503202 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.503207    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:18:38.546035 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.545994    2573 generic.go:356] "Generic (PLEG): container finished" podID="83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" containerID="7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86" exitCode=0
Jul 15 14:18:38.546204 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.546046    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"
Jul 15 14:18:38.546204 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.546066    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57" event={"ID":"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b","Type":"ContainerDied","Data":"7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86"}
Jul 15 14:18:38.546204 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.546110    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57" event={"ID":"83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b","Type":"ContainerDied","Data":"9d0e7e5158ea615747275fd5159626adbe6147665c271041743420b361df1e7f"}
Jul 15 14:18:38.546204 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.546132    2573 scope.go:117] "RemoveContainer" containerID="7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86"
Jul 15 14:18:38.555653 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.555632    2573 scope.go:117] "RemoveContainer" containerID="7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86"
Jul 15 14:18:38.555981 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:18:38.555960    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86\": container with ID starting with 7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86 not found: ID does not exist" containerID="7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86"
Jul 15 14:18:38.556056 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.555995    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86"} err="failed to get container status \"7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86\": rpc error: code = NotFound desc = could not find container \"7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86\": container with ID starting with 7ecc74bba09d457768cb8bc1fcd914c1e8e181ead62efe57d2703b4d20c73e86 not found: ID does not exist"
Jul 15 14:18:38.570725 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.570690    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"]
Jul 15 14:18:38.575705 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:38.575677    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d2f55b-56d4d5ccc-tgq57"]
Jul 15 14:18:39.406823 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:39.406789    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" path="/var/lib/kubelet/pods/83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b/volumes"
Jul 15 14:18:45.961050 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.961007    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"]
Jul 15 14:18:45.961619 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.961598    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f41902-068a-457f-a401-eb7bb64ec931" containerName="revoke-keys"
Jul 15 14:18:45.961699 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.961622    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f41902-068a-457f-a401-eb7bb64ec931" containerName="revoke-keys"
Jul 15 14:18:45.961699 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.961659    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" containerName="maas-api"
Jul 15 14:18:45.961699 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.961670    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" containerName="maas-api"
Jul 15 14:18:45.961844 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.961778    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="83deffe9-b7c4-4e9b-bc3c-e79fb7bea12b" containerName="maas-api"
Jul 15 14:18:45.961844 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.961796    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="90f41902-068a-457f-a401-eb7bb64ec931" containerName="revoke-keys"
Jul 15 14:18:45.966510 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.966482    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:45.969969 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.969937    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:18:45.970139 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.969937    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:18:45.970948 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.970909    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-dd81e643-serving-cert\""
Jul 15 14:18:45.970948 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.970938    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:18:45.981162 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:45.981126    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"]
Jul 15 14:18:46.073097 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.073047    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbhwj\" (UniqueName: \"kubernetes.io/projected/a1a63fe0-1838-4382-aaed-7784651c8f37-kube-api-access-fbhwj\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.073281 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.073123    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1a63fe0-1838-4382-aaed-7784651c8f37-maas-api-tls\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.073281 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.073192    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a63fe0-1838-4382-aaed-7784651c8f37-openshift-service-ca\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.174104 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.174064    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a63fe0-1838-4382-aaed-7784651c8f37-openshift-service-ca\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.174104 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.174111    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbhwj\" (UniqueName: \"kubernetes.io/projected/a1a63fe0-1838-4382-aaed-7784651c8f37-kube-api-access-fbhwj\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.174307 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.174160    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1a63fe0-1838-4382-aaed-7784651c8f37-maas-api-tls\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.174890 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.174862    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a63fe0-1838-4382-aaed-7784651c8f37-openshift-service-ca\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.176678 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.176653    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1a63fe0-1838-4382-aaed-7784651c8f37-maas-api-tls\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.187112 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.187082    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbhwj\" (UniqueName: \"kubernetes.io/projected/a1a63fe0-1838-4382-aaed-7784651c8f37-kube-api-access-fbhwj\") pod \"maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.279546 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.279492    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:46.427898 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.427859    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"]
Jul 15 14:18:46.429923 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:18:46.429898    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a63fe0_1838_4382_aaed_7784651c8f37.slice/crio-87ebfbf3e305ea58be239e0ba233cba6985badf21a6be90d6a58e18bffae13be WatchSource:0}: Error finding container 87ebfbf3e305ea58be239e0ba233cba6985badf21a6be90d6a58e18bffae13be: Status 404 returned error can't find the container with id 87ebfbf3e305ea58be239e0ba233cba6985badf21a6be90d6a58e18bffae13be
Jul 15 14:18:46.590872 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:46.590784    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7" event={"ID":"a1a63fe0-1838-4382-aaed-7784651c8f37","Type":"ContainerStarted","Data":"87ebfbf3e305ea58be239e0ba233cba6985badf21a6be90d6a58e18bffae13be"}
Jul 15 14:18:47.597764 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:47.597723    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7" event={"ID":"a1a63fe0-1838-4382-aaed-7784651c8f37","Type":"ContainerStarted","Data":"47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691"}
Jul 15 14:18:47.598171 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:47.597808    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:47.644778 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:47.644728    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7" podStartSLOduration=2.267157904 podStartE2EDuration="2.644709532s" totalImagesPullingTime="377.551628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:18:47.644128414 +0000 UTC m=+1648.763933996" watchObservedRunningTime="2026-07-15 14:18:47.644709532 +0000 UTC m=+1648.764515112"
Jul 15 14:18:50.970631 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:50.970592    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"]
Jul 15 14:18:50.974193 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:50.974172    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"
Jul 15 14:18:50.979512 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:50.979487    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\""
Jul 15 14:18:51.000001 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:50.999962    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"]
Jul 15 14:18:51.122462 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:51.122423    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3950ec4-1327-4642-97d1-40da158b9abd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk\" (UID: \"b3950ec4-1327-4642-97d1-40da158b9abd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"
Jul 15 14:18:51.223651 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:51.223573    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3950ec4-1327-4642-97d1-40da158b9abd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk\" (UID: \"b3950ec4-1327-4642-97d1-40da158b9abd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"
Jul 15 14:18:51.224225 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:51.224202    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3950ec4-1327-4642-97d1-40da158b9abd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk\" (UID: \"b3950ec4-1327-4642-97d1-40da158b9abd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"
Jul 15 14:18:51.285009 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:51.284960    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"
Jul 15 14:18:51.435250 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:51.435212    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"]
Jul 15 14:18:51.438381 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:18:51.438328    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3950ec4_1327_4642_97d1_40da158b9abd.slice/crio-353fc33e31826c3fe72960815b6f2e1d8b24e02cc755ad610f9069979b96d206 WatchSource:0}: Error finding container 353fc33e31826c3fe72960815b6f2e1d8b24e02cc755ad610f9069979b96d206: Status 404 returned error can't find the container with id 353fc33e31826c3fe72960815b6f2e1d8b24e02cc755ad610f9069979b96d206
Jul 15 14:18:51.620549 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:51.620513    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk" event={"ID":"b3950ec4-1327-4642-97d1-40da158b9abd","Type":"ContainerStarted","Data":"353fc33e31826c3fe72960815b6f2e1d8b24e02cc755ad610f9069979b96d206"}
Jul 15 14:18:52.626860 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:52.626830    2573 generic.go:356] "Generic (PLEG): container finished" podID="b3950ec4-1327-4642-97d1-40da158b9abd" containerID="ac9165aaec1c2c4faf001ec03c9c397e9d6735b945691c104286771051601a42" exitCode=7
Jul 15 14:18:52.627258 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:52.626911    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk" event={"ID":"b3950ec4-1327-4642-97d1-40da158b9abd","Type":"ContainerDied","Data":"ac9165aaec1c2c4faf001ec03c9c397e9d6735b945691c104286771051601a42"}
Jul 15 14:18:52.627258 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:52.627091    2573 scope.go:117] "RemoveContainer" containerID="ac9165aaec1c2c4faf001ec03c9c397e9d6735b945691c104286771051601a42"
Jul 15 14:18:53.609560 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:53.609477    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:18:53.634587 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:53.634553    2573 generic.go:356] "Generic (PLEG): container finished" podID="b3950ec4-1327-4642-97d1-40da158b9abd" containerID="a04830f45bad0cbf4036b196f319b0844947d910f054ed8c747a331639bc41ca" exitCode=7
Jul 15 14:18:53.634974 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:53.634641    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk" event={"ID":"b3950ec4-1327-4642-97d1-40da158b9abd","Type":"ContainerDied","Data":"a04830f45bad0cbf4036b196f319b0844947d910f054ed8c747a331639bc41ca"}
Jul 15 14:18:53.634974 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:53.634702    2573 scope.go:117] "RemoveContainer" containerID="ac9165aaec1c2c4faf001ec03c9c397e9d6735b945691c104286771051601a42"
Jul 15 14:18:53.634974 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:53.634926    2573 scope.go:117] "RemoveContainer" containerID="a04830f45bad0cbf4036b196f319b0844947d910f054ed8c747a331639bc41ca"
Jul 15 14:18:53.635190 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:18:53.635168    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk_odh-ai-gateway-infra(b3950ec4-1327-4642-97d1-40da158b9abd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk" podUID="b3950ec4-1327-4642-97d1-40da158b9abd"
Jul 15 14:18:54.642397 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:18:54.642368    2573 scope.go:117] "RemoveContainer" containerID="a04830f45bad0cbf4036b196f319b0844947d910f054ed8c747a331639bc41ca"
Jul 15 14:18:54.642762 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:18:54.642600    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk_odh-ai-gateway-infra(b3950ec4-1327-4642-97d1-40da158b9abd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk" podUID="b3950ec4-1327-4642-97d1-40da158b9abd"
Jul 15 14:19:07.400757 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:07.400717    2573 scope.go:117] "RemoveContainer" containerID="a04830f45bad0cbf4036b196f319b0844947d910f054ed8c747a331639bc41ca"
Jul 15 14:19:08.722210 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:08.722178    2573 generic.go:356] "Generic (PLEG): container finished" podID="b3950ec4-1327-4642-97d1-40da158b9abd" containerID="d8805b57f1817b082e31149ebbdfd0445429f40b2936abe957dafafdd919ae4b" exitCode=0
Jul 15 14:19:08.722681 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:08.722234    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk" event={"ID":"b3950ec4-1327-4642-97d1-40da158b9abd","Type":"ContainerDied","Data":"d8805b57f1817b082e31149ebbdfd0445429f40b2936abe957dafafdd919ae4b"}
Jul 15 14:19:08.722681 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:08.722304    2573 scope.go:117] "RemoveContainer" containerID="a04830f45bad0cbf4036b196f319b0844947d910f054ed8c747a331639bc41ca"
Jul 15 14:19:09.789064 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:09.789026    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"]
Jul 15 14:19:09.871012 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:09.870981    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"
Jul 15 14:19:09.901171 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:09.901137    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3950ec4-1327-4642-97d1-40da158b9abd-maas-api-service-ca\") pod \"b3950ec4-1327-4642-97d1-40da158b9abd\" (UID: \"b3950ec4-1327-4642-97d1-40da158b9abd\") "
Jul 15 14:19:09.901557 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:09.901534    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3950ec4-1327-4642-97d1-40da158b9abd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3950ec4-1327-4642-97d1-40da158b9abd" (UID: "b3950ec4-1327-4642-97d1-40da158b9abd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:19:10.002010 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:10.001919    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3950ec4-1327-4642-97d1-40da158b9abd-maas-api-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:19:10.735321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:10.735288    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"
Jul 15 14:19:10.735321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:10.735298    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk" event={"ID":"b3950ec4-1327-4642-97d1-40da158b9abd","Type":"ContainerDied","Data":"353fc33e31826c3fe72960815b6f2e1d8b24e02cc755ad610f9069979b96d206"}
Jul 15 14:19:10.735550 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:10.735375    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="353fc33e31826c3fe72960815b6f2e1d8b24e02cc755ad610f9069979b96d206"
Jul 15 14:19:10.773003 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:10.772966    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"]
Jul 15 14:19:10.778953 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:10.778922    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-jwjzk"]
Jul 15 14:19:11.407646 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:11.407612    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" path="/var/lib/kubelet/pods/b3950ec4-1327-4642-97d1-40da158b9abd/volumes"
Jul 15 14:19:41.040500 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040420    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"]
Jul 15 14:19:41.040941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040796    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:19:41.040941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040808    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:19:41.040941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040820    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:19:41.040941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040827    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:19:41.040941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040835    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:19:41.040941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040840    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:19:41.040941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040909    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:19:41.040941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.040923    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:19:41.044063 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.044044    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"
Jul 15 14:19:41.048813 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.048792    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\""
Jul 15 14:19:41.063793 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.063758    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"]
Jul 15 14:19:41.202495 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.202450    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece82a83-93ad-4f7a-a7c6-e86ee1a2608a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf\" (UID: \"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"
Jul 15 14:19:41.303605 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.303514    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece82a83-93ad-4f7a-a7c6-e86ee1a2608a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf\" (UID: \"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"
Jul 15 14:19:41.304152 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.304133    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece82a83-93ad-4f7a-a7c6-e86ee1a2608a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf\" (UID: \"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"
Jul 15 14:19:41.354148 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.354109    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"
Jul 15 14:19:41.494005 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.493980    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"]
Jul 15 14:19:41.496189 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:19:41.496159    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece82a83_93ad_4f7a_a7c6_e86ee1a2608a.slice/crio-ea32e6fa5dd1d6f502b705b0fa58950ed5cbb388d587c840dd805b0e8248f9e7 WatchSource:0}: Error finding container ea32e6fa5dd1d6f502b705b0fa58950ed5cbb388d587c840dd805b0e8248f9e7: Status 404 returned error can't find the container with id ea32e6fa5dd1d6f502b705b0fa58950ed5cbb388d587c840dd805b0e8248f9e7
Jul 15 14:19:41.896995 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.896961    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf" event={"ID":"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a","Type":"ContainerStarted","Data":"0f9ef7153387e32fe60dae804a26161c081d431d45c53fbaffb93da1f35b6999"}
Jul 15 14:19:41.897173 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.897022    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf" event={"ID":"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a","Type":"ContainerStarted","Data":"ea32e6fa5dd1d6f502b705b0fa58950ed5cbb388d587c840dd805b0e8248f9e7"}
Jul 15 14:19:41.917512 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:41.917455    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf" podStartSLOduration=0.917436432 podStartE2EDuration="917.436432ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:19:41.916921009 +0000 UTC m=+1703.036726589" watchObservedRunningTime="2026-07-15 14:19:41.917436432 +0000 UTC m=+1703.037242013"
Jul 15 14:19:42.903003 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:42.902971    2573 generic.go:356] "Generic (PLEG): container finished" podID="ece82a83-93ad-4f7a-a7c6-e86ee1a2608a" containerID="0f9ef7153387e32fe60dae804a26161c081d431d45c53fbaffb93da1f35b6999" exitCode=0
Jul 15 14:19:42.903407 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:42.903047    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf" event={"ID":"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a","Type":"ContainerDied","Data":"0f9ef7153387e32fe60dae804a26161c081d431d45c53fbaffb93da1f35b6999"}
Jul 15 14:19:44.039031 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:44.039003    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"
Jul 15 14:19:44.128911 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:44.128871    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece82a83-93ad-4f7a-a7c6-e86ee1a2608a-maas-api-service-ca\") pod \"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a\" (UID: \"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a\") "
Jul 15 14:19:44.129267 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:44.129237    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ece82a83-93ad-4f7a-a7c6-e86ee1a2608a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ece82a83-93ad-4f7a-a7c6-e86ee1a2608a" (UID: "ece82a83-93ad-4f7a-a7c6-e86ee1a2608a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:19:44.229917 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:44.229832    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece82a83-93ad-4f7a-a7c6-e86ee1a2608a-maas-api-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:19:44.913585 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:44.913552    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf" event={"ID":"ece82a83-93ad-4f7a-a7c6-e86ee1a2608a","Type":"ContainerDied","Data":"ea32e6fa5dd1d6f502b705b0fa58950ed5cbb388d587c840dd805b0e8248f9e7"}
Jul 15 14:19:44.913585 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:44.913586    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf"
Jul 15 14:19:44.913793 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:44.913607    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea32e6fa5dd1d6f502b705b0fa58950ed5cbb388d587c840dd805b0e8248f9e7"
Jul 15 14:19:51.566212 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.566176    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"]
Jul 15 14:19:51.566740 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.566467    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7" podUID="a1a63fe0-1838-4382-aaed-7784651c8f37" containerName="maas-api" containerID="cri-o://47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691" gracePeriod=30
Jul 15 14:19:51.819320 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.819248    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:19:51.948809 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.948772    2573 generic.go:356] "Generic (PLEG): container finished" podID="a1a63fe0-1838-4382-aaed-7784651c8f37" containerID="47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691" exitCode=0
Jul 15 14:19:51.949010 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.948825    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"
Jul 15 14:19:51.949010 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.948858    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7" event={"ID":"a1a63fe0-1838-4382-aaed-7784651c8f37","Type":"ContainerDied","Data":"47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691"}
Jul 15 14:19:51.949010 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.948908    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7" event={"ID":"a1a63fe0-1838-4382-aaed-7784651c8f37","Type":"ContainerDied","Data":"87ebfbf3e305ea58be239e0ba233cba6985badf21a6be90d6a58e18bffae13be"}
Jul 15 14:19:51.949010 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.948930    2573 scope.go:117] "RemoveContainer" containerID="47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691"
Jul 15 14:19:51.957355 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.957314    2573 scope.go:117] "RemoveContainer" containerID="47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691"
Jul 15 14:19:51.957637 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:19:51.957616    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691\": container with ID starting with 47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691 not found: ID does not exist" containerID="47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691"
Jul 15 14:19:51.957700 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:51.957647    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691"} err="failed to get container status \"47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691\": rpc error: code = NotFound desc = could not find container \"47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691\": container with ID starting with 47e038cdcc53ca39c90fa679ee9e7cc8e89d430f44ae2160e55c5bdfec07a691 not found: ID does not exist"
Jul 15 14:19:52.005049 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.005017    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a63fe0-1838-4382-aaed-7784651c8f37-openshift-service-ca\") pod \"a1a63fe0-1838-4382-aaed-7784651c8f37\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") "
Jul 15 14:19:52.005182 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.005058    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1a63fe0-1838-4382-aaed-7784651c8f37-maas-api-tls\") pod \"a1a63fe0-1838-4382-aaed-7784651c8f37\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") "
Jul 15 14:19:52.005182 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.005102    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbhwj\" (UniqueName: \"kubernetes.io/projected/a1a63fe0-1838-4382-aaed-7784651c8f37-kube-api-access-fbhwj\") pod \"a1a63fe0-1838-4382-aaed-7784651c8f37\" (UID: \"a1a63fe0-1838-4382-aaed-7784651c8f37\") "
Jul 15 14:19:52.005511 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.005482    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a63fe0-1838-4382-aaed-7784651c8f37-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1a63fe0-1838-4382-aaed-7784651c8f37" (UID: "a1a63fe0-1838-4382-aaed-7784651c8f37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:19:52.007257 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.007227    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a63fe0-1838-4382-aaed-7784651c8f37-kube-api-access-fbhwj" (OuterVolumeSpecName: "kube-api-access-fbhwj") pod "a1a63fe0-1838-4382-aaed-7784651c8f37" (UID: "a1a63fe0-1838-4382-aaed-7784651c8f37"). InnerVolumeSpecName "kube-api-access-fbhwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:19:52.007257 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.007239    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1a63fe0-1838-4382-aaed-7784651c8f37-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1a63fe0-1838-4382-aaed-7784651c8f37" (UID: "a1a63fe0-1838-4382-aaed-7784651c8f37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:19:52.106988 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.106881    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a63fe0-1838-4382-aaed-7784651c8f37-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:19:52.106988 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.106928    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1a63fe0-1838-4382-aaed-7784651c8f37-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:19:52.106988 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.106938    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbhwj\" (UniqueName: \"kubernetes.io/projected/a1a63fe0-1838-4382-aaed-7784651c8f37-kube-api-access-fbhwj\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:19:52.284074 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.284039    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"]
Jul 15 14:19:52.296523 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:52.296488    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd81e643-65b44f4b46-9j7x7"]
Jul 15 14:19:53.407384 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:19:53.407325    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1a63fe0-1838-4382-aaed-7784651c8f37" path="/var/lib/kubelet/pods/a1a63fe0-1838-4382-aaed-7784651c8f37/volumes"
Jul 15 14:20:09.021108 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.021066    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"]
Jul 15 14:20:09.021658 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.021638    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1a63fe0-1838-4382-aaed-7784651c8f37" containerName="maas-api"
Jul 15 14:20:09.021738 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.021662    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1a63fe0-1838-4382-aaed-7784651c8f37" containerName="maas-api"
Jul 15 14:20:09.021738 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.021702    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece82a83-93ad-4f7a-a7c6-e86ee1a2608a" containerName="revoke-keys"
Jul 15 14:20:09.021738 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.021711    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece82a83-93ad-4f7a-a7c6-e86ee1a2608a" containerName="revoke-keys"
Jul 15 14:20:09.021878 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.021834    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece82a83-93ad-4f7a-a7c6-e86ee1a2608a" containerName="revoke-keys"
Jul 15 14:20:09.021878 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.021851    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3950ec4-1327-4642-97d1-40da158b9abd" containerName="revoke-keys"
Jul 15 14:20:09.021878 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.021867    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1a63fe0-1838-4382-aaed-7784651c8f37" containerName="maas-api"
Jul 15 14:20:09.026575 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.026552    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.029511 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.029488    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:20:09.029628 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.029489    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-55a6f882-serving-cert\""
Jul 15 14:20:09.030649 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.030629    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:20:09.030720 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.030675    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:20:09.042186 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.042154    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"]
Jul 15 14:20:09.165876 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.165842    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d446af4-e77b-41e8-a267-b977d0687bd4-openshift-service-ca\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.166061 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.165952    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.166121 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.166058    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk6sc\" (UniqueName: \"kubernetes.io/projected/4d446af4-e77b-41e8-a267-b977d0687bd4-kube-api-access-zk6sc\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.267244 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.267196    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6sc\" (UniqueName: \"kubernetes.io/projected/4d446af4-e77b-41e8-a267-b977d0687bd4-kube-api-access-zk6sc\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.267244 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.267259    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d446af4-e77b-41e8-a267-b977d0687bd4-openshift-service-ca\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.267546 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.267304    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.267546 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:20:09.267470    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-serving-cert: secret "maas-api-e2e-derive-55a6f882-serving-cert" not found
Jul 15 14:20:09.267546 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:20:09.267537    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls podName:4d446af4-e77b-41e8-a267-b977d0687bd4 nodeName:}" failed. No retries permitted until 2026-07-15 14:20:09.767517528 +0000 UTC m=+1730.887323089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls") pod "maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9" (UID: "4d446af4-e77b-41e8-a267-b977d0687bd4") : secret "maas-api-e2e-derive-55a6f882-serving-cert" not found
Jul 15 14:20:09.268010 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.267988    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d446af4-e77b-41e8-a267-b977d0687bd4-openshift-service-ca\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.278571 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.278502    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk6sc\" (UniqueName: \"kubernetes.io/projected/4d446af4-e77b-41e8-a267-b977d0687bd4-kube-api-access-zk6sc\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.773000 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.772958    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.775449 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.775416    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls\") pod \"maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:09.938273 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:09.938234    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:10.090996 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:10.090973    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"]
Jul 15 14:20:10.093237 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:20:10.093208    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d446af4_e77b_41e8_a267_b977d0687bd4.slice/crio-19c9da6397deedcab40f95ebce8c5df042e8c2ee601279987cea3daccd7df1b8 WatchSource:0}: Error finding container 19c9da6397deedcab40f95ebce8c5df042e8c2ee601279987cea3daccd7df1b8: Status 404 returned error can't find the container with id 19c9da6397deedcab40f95ebce8c5df042e8c2ee601279987cea3daccd7df1b8
Jul 15 14:20:11.049543 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:11.049502    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9" event={"ID":"4d446af4-e77b-41e8-a267-b977d0687bd4","Type":"ContainerStarted","Data":"6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1"}
Jul 15 14:20:11.049748 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:11.049560    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9" event={"ID":"4d446af4-e77b-41e8-a267-b977d0687bd4","Type":"ContainerStarted","Data":"19c9da6397deedcab40f95ebce8c5df042e8c2ee601279987cea3daccd7df1b8"}
Jul 15 14:20:11.049748 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:11.049676    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:20:11.077913 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:11.077858    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9" podStartSLOduration=2.647181589 podStartE2EDuration="3.077840246s" totalImagesPullingTime="430.658657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:20:11.075350559 +0000 UTC m=+1732.195156139" watchObservedRunningTime="2026-07-15 14:20:11.077840246 +0000 UTC m=+1732.197645827"
Jul 15 14:20:17.059434 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:20:17.059403    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:21:13.306189 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.306107    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"]
Jul 15 14:21:13.309901 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.309878    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"
Jul 15 14:21:13.312743 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.312722    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\""
Jul 15 14:21:13.317859 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.317831    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"]
Jul 15 14:21:13.400742 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.400690    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83db0a14-0e07-4d68-b232-3faa2dd36faf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55a6f882-czw7r\" (UID: \"83db0a14-0e07-4d68-b232-3faa2dd36faf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"
Jul 15 14:21:13.501491 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.501451    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83db0a14-0e07-4d68-b232-3faa2dd36faf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55a6f882-czw7r\" (UID: \"83db0a14-0e07-4d68-b232-3faa2dd36faf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"
Jul 15 14:21:13.502062 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.502031    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83db0a14-0e07-4d68-b232-3faa2dd36faf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55a6f882-czw7r\" (UID: \"83db0a14-0e07-4d68-b232-3faa2dd36faf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"
Jul 15 14:21:13.623107 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.623008    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"
Jul 15 14:21:13.771189 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:13.771160    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"]
Jul 15 14:21:13.773070 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:21:13.773043    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83db0a14_0e07_4d68_b232_3faa2dd36faf.slice/crio-d52b4b37c7f5e34f5b86a76302c74123e00df9031ef608a4922c93ab6cd1fca9 WatchSource:0}: Error finding container d52b4b37c7f5e34f5b86a76302c74123e00df9031ef608a4922c93ab6cd1fca9: Status 404 returned error can't find the container with id d52b4b37c7f5e34f5b86a76302c74123e00df9031ef608a4922c93ab6cd1fca9
Jul 15 14:21:14.377759 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:14.377721    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r" event={"ID":"83db0a14-0e07-4d68-b232-3faa2dd36faf","Type":"ContainerStarted","Data":"87ba7bc19c5dc17d64c88d2b4ae6f766732cf4a505189bb4d9f61b4b5f67b6af"}
Jul 15 14:21:14.378151 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:14.377776    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r" event={"ID":"83db0a14-0e07-4d68-b232-3faa2dd36faf","Type":"ContainerStarted","Data":"d52b4b37c7f5e34f5b86a76302c74123e00df9031ef608a4922c93ab6cd1fca9"}
Jul 15 14:21:14.397063 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:14.397003    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r" podStartSLOduration=1.396978859 podStartE2EDuration="1.396978859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:21:14.395245323 +0000 UTC m=+1795.515050896" watchObservedRunningTime="2026-07-15 14:21:14.396978859 +0000 UTC m=+1795.516784442"
Jul 15 14:21:15.384207 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:15.384172    2573 generic.go:356] "Generic (PLEG): container finished" podID="83db0a14-0e07-4d68-b232-3faa2dd36faf" containerID="87ba7bc19c5dc17d64c88d2b4ae6f766732cf4a505189bb4d9f61b4b5f67b6af" exitCode=0
Jul 15 14:21:15.384630 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:15.384235    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r" event={"ID":"83db0a14-0e07-4d68-b232-3faa2dd36faf","Type":"ContainerDied","Data":"87ba7bc19c5dc17d64c88d2b4ae6f766732cf4a505189bb4d9f61b4b5f67b6af"}
Jul 15 14:21:16.531307 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:16.531277    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"
Jul 15 14:21:16.628240 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:16.628203    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83db0a14-0e07-4d68-b232-3faa2dd36faf-maas-api-service-ca\") pod \"83db0a14-0e07-4d68-b232-3faa2dd36faf\" (UID: \"83db0a14-0e07-4d68-b232-3faa2dd36faf\") "
Jul 15 14:21:16.628599 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:16.628573    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83db0a14-0e07-4d68-b232-3faa2dd36faf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "83db0a14-0e07-4d68-b232-3faa2dd36faf" (UID: "83db0a14-0e07-4d68-b232-3faa2dd36faf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:21:16.729114 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:16.729026    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83db0a14-0e07-4d68-b232-3faa2dd36faf-maas-api-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:21:17.395810 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:17.395780    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r"
Jul 15 14:21:17.396019 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:17.395817    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55a6f882-czw7r" event={"ID":"83db0a14-0e07-4d68-b232-3faa2dd36faf","Type":"ContainerDied","Data":"d52b4b37c7f5e34f5b86a76302c74123e00df9031ef608a4922c93ab6cd1fca9"}
Jul 15 14:21:17.396019 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:17.395876    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d52b4b37c7f5e34f5b86a76302c74123e00df9031ef608a4922c93ab6cd1fca9"
Jul 15 14:21:19.473081 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:19.473045    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:21:19.475767 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:19.475739    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:21:23.788765 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:23.788720    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"]
Jul 15 14:21:23.789192 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:23.789004    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9" podUID="4d446af4-e77b-41e8-a267-b977d0687bd4" containerName="maas-api" containerID="cri-o://6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1" gracePeriod=30
Jul 15 14:21:24.053137 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.053052    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:21:24.103348 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.103312    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk6sc\" (UniqueName: \"kubernetes.io/projected/4d446af4-e77b-41e8-a267-b977d0687bd4-kube-api-access-zk6sc\") pod \"4d446af4-e77b-41e8-a267-b977d0687bd4\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") "
Jul 15 14:21:24.103569 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.103405    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d446af4-e77b-41e8-a267-b977d0687bd4-openshift-service-ca\") pod \"4d446af4-e77b-41e8-a267-b977d0687bd4\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") "
Jul 15 14:21:24.103569 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.103429    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls\") pod \"4d446af4-e77b-41e8-a267-b977d0687bd4\" (UID: \"4d446af4-e77b-41e8-a267-b977d0687bd4\") "
Jul 15 14:21:24.103773 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.103746    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d446af4-e77b-41e8-a267-b977d0687bd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d446af4-e77b-41e8-a267-b977d0687bd4" (UID: "4d446af4-e77b-41e8-a267-b977d0687bd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:21:24.105645 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.105616    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d446af4-e77b-41e8-a267-b977d0687bd4-kube-api-access-zk6sc" (OuterVolumeSpecName: "kube-api-access-zk6sc") pod "4d446af4-e77b-41e8-a267-b977d0687bd4" (UID: "4d446af4-e77b-41e8-a267-b977d0687bd4"). InnerVolumeSpecName "kube-api-access-zk6sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:21:24.105734 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.105617    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d446af4-e77b-41e8-a267-b977d0687bd4" (UID: "4d446af4-e77b-41e8-a267-b977d0687bd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:21:24.204767 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.204722    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk6sc\" (UniqueName: \"kubernetes.io/projected/4d446af4-e77b-41e8-a267-b977d0687bd4-kube-api-access-zk6sc\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:21:24.204767 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.204758    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d446af4-e77b-41e8-a267-b977d0687bd4-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:21:24.204767 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.204768    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d446af4-e77b-41e8-a267-b977d0687bd4-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:21:24.434581 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.434490    2573 generic.go:356] "Generic (PLEG): container finished" podID="4d446af4-e77b-41e8-a267-b977d0687bd4" containerID="6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1" exitCode=0
Jul 15 14:21:24.434581 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.434549    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"
Jul 15 14:21:24.434811 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.434573    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9" event={"ID":"4d446af4-e77b-41e8-a267-b977d0687bd4","Type":"ContainerDied","Data":"6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1"}
Jul 15 14:21:24.434811 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.434637    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9" event={"ID":"4d446af4-e77b-41e8-a267-b977d0687bd4","Type":"ContainerDied","Data":"19c9da6397deedcab40f95ebce8c5df042e8c2ee601279987cea3daccd7df1b8"}
Jul 15 14:21:24.434811 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.434665    2573 scope.go:117] "RemoveContainer" containerID="6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1"
Jul 15 14:21:24.444275 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.444253    2573 scope.go:117] "RemoveContainer" containerID="6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1"
Jul 15 14:21:24.444597 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:21:24.444573    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1\": container with ID starting with 6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1 not found: ID does not exist" containerID="6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1"
Jul 15 14:21:24.444645 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.444606    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1"} err="failed to get container status \"6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1\": rpc error: code = NotFound desc = could not find container \"6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1\": container with ID starting with 6622dfa54ebd1fb08a818bc3c0c9da9d0bd32d4bf57ac07f0559246059c1cca1 not found: ID does not exist"
Jul 15 14:21:24.457892 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.457838    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"]
Jul 15 14:21:24.464213 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:24.464177    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55a6f882-6ddbdc884f-lf2v9"]
Jul 15 14:21:25.406992 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:21:25.406955    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d446af4-e77b-41e8-a267-b977d0687bd4" path="/var/lib/kubelet/pods/4d446af4-e77b-41e8-a267-b977d0687bd4/volumes"
Jul 15 14:23:20.173578 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.173542    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"]
Jul 15 14:23:20.176102 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.173979    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d446af4-e77b-41e8-a267-b977d0687bd4" containerName="maas-api"
Jul 15 14:23:20.176102 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.173992    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d446af4-e77b-41e8-a267-b977d0687bd4" containerName="maas-api"
Jul 15 14:23:20.176102 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.174004    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83db0a14-0e07-4d68-b232-3faa2dd36faf" containerName="revoke-keys"
Jul 15 14:23:20.176102 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.174011    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="83db0a14-0e07-4d68-b232-3faa2dd36faf" containerName="revoke-keys"
Jul 15 14:23:20.176102 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.174085    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d446af4-e77b-41e8-a267-b977d0687bd4" containerName="maas-api"
Jul 15 14:23:20.176102 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.174100    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="83db0a14-0e07-4d68-b232-3faa2dd36faf" containerName="revoke-keys"
Jul 15 14:23:20.177119 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.177101    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.180169 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.180145    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-625773fa-serving-cert\""
Jul 15 14:23:20.180305 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.180214    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:23:20.181472 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.181448    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:23:20.181575 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.181451    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:23:20.197840 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.197812    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"]
Jul 15 14:23:20.300058 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.300018    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszlt\" (UniqueName: \"kubernetes.io/projected/2c0f3147-69ec-4896-bd60-05f139e895ed-kube-api-access-rszlt\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.300228 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.300073    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0f3147-69ec-4896-bd60-05f139e895ed-maas-api-tls\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.300228 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.300139    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0f3147-69ec-4896-bd60-05f139e895ed-openshift-service-ca\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.401322 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.401286    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszlt\" (UniqueName: \"kubernetes.io/projected/2c0f3147-69ec-4896-bd60-05f139e895ed-kube-api-access-rszlt\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.401497 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.401373    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0f3147-69ec-4896-bd60-05f139e895ed-maas-api-tls\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.401497 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.401421    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0f3147-69ec-4896-bd60-05f139e895ed-openshift-service-ca\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.402129 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.402096    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0f3147-69ec-4896-bd60-05f139e895ed-openshift-service-ca\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.403912 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.403889    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0f3147-69ec-4896-bd60-05f139e895ed-maas-api-tls\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.410450 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.410425    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszlt\" (UniqueName: \"kubernetes.io/projected/2c0f3147-69ec-4896-bd60-05f139e895ed-kube-api-access-rszlt\") pod \"maas-api-e2e-mt-625773fa-796f584bd5-2xg44\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.488141 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.488049    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:20.635840 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.635808    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"]
Jul 15 14:23:20.638995 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:23:20.638960    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0f3147_69ec_4896_bd60_05f139e895ed.slice/crio-0c81e7acdae2a1cffc9e0e333e482b78dcf167f6de589c83db3be145d8ddfda7 WatchSource:0}: Error finding container 0c81e7acdae2a1cffc9e0e333e482b78dcf167f6de589c83db3be145d8ddfda7: Status 404 returned error can't find the container with id 0c81e7acdae2a1cffc9e0e333e482b78dcf167f6de589c83db3be145d8ddfda7
Jul 15 14:23:20.645943 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:20.643118    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 14:23:21.044983 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:21.044944    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44" event={"ID":"2c0f3147-69ec-4896-bd60-05f139e895ed","Type":"ContainerStarted","Data":"0c81e7acdae2a1cffc9e0e333e482b78dcf167f6de589c83db3be145d8ddfda7"}
Jul 15 14:23:22.051808 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:22.051760    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44" event={"ID":"2c0f3147-69ec-4896-bd60-05f139e895ed","Type":"ContainerStarted","Data":"07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b"}
Jul 15 14:23:22.052220 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:22.051926    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:22.079269 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:22.079217    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44" podStartSLOduration=1.565168693 podStartE2EDuration="2.079199885s" totalImagesPullingTime="514.031192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:23:22.077208685 +0000 UTC m=+1923.197014279" watchObservedRunningTime="2026-07-15 14:23:22.079199885 +0000 UTC m=+1923.199005465"
Jul 15 14:23:28.062960 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:28.062924    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:32.864825 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:32.864787    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"]
Jul 15 14:23:32.868850 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:32.868816    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"
Jul 15 14:23:32.871775 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:32.871748    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\""
Jul 15 14:23:32.877697 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:32.877667    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"]
Jul 15 14:23:33.024523 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:33.024483    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71216994-5e7c-4a1c-81a0-22f149cb2cff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-625773fa-wmkd2\" (UID: \"71216994-5e7c-4a1c-81a0-22f149cb2cff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"
Jul 15 14:23:33.126171 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:33.126069    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71216994-5e7c-4a1c-81a0-22f149cb2cff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-625773fa-wmkd2\" (UID: \"71216994-5e7c-4a1c-81a0-22f149cb2cff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"
Jul 15 14:23:33.126787 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:33.126766    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71216994-5e7c-4a1c-81a0-22f149cb2cff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-625773fa-wmkd2\" (UID: \"71216994-5e7c-4a1c-81a0-22f149cb2cff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"
Jul 15 14:23:33.181909 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:33.181873    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"
Jul 15 14:23:33.332321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:33.332292    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"]
Jul 15 14:23:33.334053 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:23:33.334014    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71216994_5e7c_4a1c_81a0_22f149cb2cff.slice/crio-d6677bf0712a99be5301d7fb9b8adc7f79953e5cadd78d46a2bd85fc4be5ff26 WatchSource:0}: Error finding container d6677bf0712a99be5301d7fb9b8adc7f79953e5cadd78d46a2bd85fc4be5ff26: Status 404 returned error can't find the container with id d6677bf0712a99be5301d7fb9b8adc7f79953e5cadd78d46a2bd85fc4be5ff26
Jul 15 14:23:34.114371 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:34.114308    2573 generic.go:356] "Generic (PLEG): container finished" podID="71216994-5e7c-4a1c-81a0-22f149cb2cff" containerID="e3feabfe32820cbc84e2357e2d74611e719d63060d0ff974ca7d45c3426ea7a9" exitCode=0
Jul 15 14:23:34.114764 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:34.114393    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2" event={"ID":"71216994-5e7c-4a1c-81a0-22f149cb2cff","Type":"ContainerDied","Data":"e3feabfe32820cbc84e2357e2d74611e719d63060d0ff974ca7d45c3426ea7a9"}
Jul 15 14:23:34.114764 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:34.114448    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2" event={"ID":"71216994-5e7c-4a1c-81a0-22f149cb2cff","Type":"ContainerStarted","Data":"d6677bf0712a99be5301d7fb9b8adc7f79953e5cadd78d46a2bd85fc4be5ff26"}
Jul 15 14:23:35.259834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:35.259803    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"
Jul 15 14:23:35.348623 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:35.348585    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71216994-5e7c-4a1c-81a0-22f149cb2cff-maas-api-service-ca\") pod \"71216994-5e7c-4a1c-81a0-22f149cb2cff\" (UID: \"71216994-5e7c-4a1c-81a0-22f149cb2cff\") "
Jul 15 14:23:35.348975 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:35.348950    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71216994-5e7c-4a1c-81a0-22f149cb2cff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71216994-5e7c-4a1c-81a0-22f149cb2cff" (UID: "71216994-5e7c-4a1c-81a0-22f149cb2cff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:23:35.449576 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:35.449535    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71216994-5e7c-4a1c-81a0-22f149cb2cff-maas-api-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:23:36.125940 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:36.125907    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2"
Jul 15 14:23:36.126104 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:36.125939    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-625773fa-wmkd2" event={"ID":"71216994-5e7c-4a1c-81a0-22f149cb2cff","Type":"ContainerDied","Data":"d6677bf0712a99be5301d7fb9b8adc7f79953e5cadd78d46a2bd85fc4be5ff26"}
Jul 15 14:23:36.126104 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:36.125987    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6677bf0712a99be5301d7fb9b8adc7f79953e5cadd78d46a2bd85fc4be5ff26"
Jul 15 14:23:43.410059 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.410011    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"]
Jul 15 14:23:43.410592 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.410277    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44" podUID="2c0f3147-69ec-4896-bd60-05f139e895ed" containerName="maas-api" containerID="cri-o://07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b" gracePeriod=30
Jul 15 14:23:43.676698 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.676609    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:43.831893 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.831851    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0f3147-69ec-4896-bd60-05f139e895ed-openshift-service-ca\") pod \"2c0f3147-69ec-4896-bd60-05f139e895ed\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") "
Jul 15 14:23:43.832069 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.832011    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0f3147-69ec-4896-bd60-05f139e895ed-maas-api-tls\") pod \"2c0f3147-69ec-4896-bd60-05f139e895ed\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") "
Jul 15 14:23:43.832069 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.832041    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rszlt\" (UniqueName: \"kubernetes.io/projected/2c0f3147-69ec-4896-bd60-05f139e895ed-kube-api-access-rszlt\") pod \"2c0f3147-69ec-4896-bd60-05f139e895ed\" (UID: \"2c0f3147-69ec-4896-bd60-05f139e895ed\") "
Jul 15 14:23:43.832246 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.832220    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c0f3147-69ec-4896-bd60-05f139e895ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c0f3147-69ec-4896-bd60-05f139e895ed" (UID: "2c0f3147-69ec-4896-bd60-05f139e895ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:23:43.834233 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.834200    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0f3147-69ec-4896-bd60-05f139e895ed-kube-api-access-rszlt" (OuterVolumeSpecName: "kube-api-access-rszlt") pod "2c0f3147-69ec-4896-bd60-05f139e895ed" (UID: "2c0f3147-69ec-4896-bd60-05f139e895ed"). InnerVolumeSpecName "kube-api-access-rszlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:23:43.834360 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.834234    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c0f3147-69ec-4896-bd60-05f139e895ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c0f3147-69ec-4896-bd60-05f139e895ed" (UID: "2c0f3147-69ec-4896-bd60-05f139e895ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:23:43.933688 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.933601    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c0f3147-69ec-4896-bd60-05f139e895ed-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:23:43.933688 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.933631    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rszlt\" (UniqueName: \"kubernetes.io/projected/2c0f3147-69ec-4896-bd60-05f139e895ed-kube-api-access-rszlt\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:23:43.933688 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:43.933641    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0f3147-69ec-4896-bd60-05f139e895ed-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:23:44.169627 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.169587    2573 generic.go:356] "Generic (PLEG): container finished" podID="2c0f3147-69ec-4896-bd60-05f139e895ed" containerID="07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b" exitCode=0
Jul 15 14:23:44.169800 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.169638    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"
Jul 15 14:23:44.169800 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.169674    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44" event={"ID":"2c0f3147-69ec-4896-bd60-05f139e895ed","Type":"ContainerDied","Data":"07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b"}
Jul 15 14:23:44.169800 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.169737    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44" event={"ID":"2c0f3147-69ec-4896-bd60-05f139e895ed","Type":"ContainerDied","Data":"0c81e7acdae2a1cffc9e0e333e482b78dcf167f6de589c83db3be145d8ddfda7"}
Jul 15 14:23:44.169800 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.169762    2573 scope.go:117] "RemoveContainer" containerID="07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b"
Jul 15 14:23:44.179625 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.179605    2573 scope.go:117] "RemoveContainer" containerID="07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b"
Jul 15 14:23:44.179945 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:23:44.179926    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b\": container with ID starting with 07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b not found: ID does not exist" containerID="07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b"
Jul 15 14:23:44.180029 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.179966    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b"} err="failed to get container status \"07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b\": rpc error: code = NotFound desc = could not find container \"07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b\": container with ID starting with 07e8f2121267e56cfdfd26aa8dc5f49136e22a796805fa769fcd5bf09f6e118b not found: ID does not exist"
Jul 15 14:23:44.194483 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.194397    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"]
Jul 15 14:23:44.200577 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:44.200541    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-625773fa-796f584bd5-2xg44"]
Jul 15 14:23:45.406583 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:23:45.406537    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c0f3147-69ec-4896-bd60-05f139e895ed" path="/var/lib/kubelet/pods/2c0f3147-69ec-4896-bd60-05f139e895ed/volumes"
Jul 15 14:24:22.473578 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.473538    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"]
Jul 15 14:24:22.474000 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.473985    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71216994-5e7c-4a1c-81a0-22f149cb2cff" containerName="revoke-keys"
Jul 15 14:24:22.474048 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.474002    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="71216994-5e7c-4a1c-81a0-22f149cb2cff" containerName="revoke-keys"
Jul 15 14:24:22.474048 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.474016    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0f3147-69ec-4896-bd60-05f139e895ed" containerName="maas-api"
Jul 15 14:24:22.474048 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.474022    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0f3147-69ec-4896-bd60-05f139e895ed" containerName="maas-api"
Jul 15 14:24:22.474159 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.474095    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="71216994-5e7c-4a1c-81a0-22f149cb2cff" containerName="revoke-keys"
Jul 15 14:24:22.474159 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.474110    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c0f3147-69ec-4896-bd60-05f139e895ed" containerName="maas-api"
Jul 15 14:24:22.478693 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.478667    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:22.482296 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.482259    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9e73fa92-serving-cert\""
Jul 15 14:24:22.482477 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.482293    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:24:22.483427 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.483406    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:24:22.483493 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.483423    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:24:22.495087 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.495056    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"]
Jul 15 14:24:22.590155 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.590116    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb3e082-e7c2-4dad-9104-fefd6d261ad1-openshift-service-ca\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:22.590394 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.590200    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:22.590394 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.590309    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q8qk\" (UniqueName: \"kubernetes.io/projected/efb3e082-e7c2-4dad-9104-fefd6d261ad1-kube-api-access-2q8qk\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:22.691620 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.691576    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb3e082-e7c2-4dad-9104-fefd6d261ad1-openshift-service-ca\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:22.691805 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.691660    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:22.691805 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.691733    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q8qk\" (UniqueName: \"kubernetes.io/projected/efb3e082-e7c2-4dad-9104-fefd6d261ad1-kube-api-access-2q8qk\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:22.691886 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:24:22.691812    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-serving-cert: secret "maas-api-e2e-mt-9e73fa92-serving-cert" not found
Jul 15 14:24:22.691921 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:24:22.691893    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls podName:efb3e082-e7c2-4dad-9104-fefd6d261ad1 nodeName:}" failed. No retries permitted until 2026-07-15 14:24:23.191875316 +0000 UTC m=+1984.311680878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls") pod "maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls" (UID: "efb3e082-e7c2-4dad-9104-fefd6d261ad1") : secret "maas-api-e2e-mt-9e73fa92-serving-cert" not found
Jul 15 14:24:22.692276 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.692256    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb3e082-e7c2-4dad-9104-fefd6d261ad1-openshift-service-ca\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:22.704413 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:22.704380    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q8qk\" (UniqueName: \"kubernetes.io/projected/efb3e082-e7c2-4dad-9104-fefd6d261ad1-kube-api-access-2q8qk\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:23.196498 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:23.196456    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:23.199055 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:23.199021    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls\") pod \"maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:23.390165 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:23.390123    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:23.539946 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:23.539913    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"]
Jul 15 14:24:23.542187 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:24:23.542160    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb3e082_e7c2_4dad_9104_fefd6d261ad1.slice/crio-aa01ac932b69deec33970dd09e88883fa56d6c606858fc0c6c0b179de7c8fdac WatchSource:0}: Error finding container aa01ac932b69deec33970dd09e88883fa56d6c606858fc0c6c0b179de7c8fdac: Status 404 returned error can't find the container with id aa01ac932b69deec33970dd09e88883fa56d6c606858fc0c6c0b179de7c8fdac
Jul 15 14:24:24.383992 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:24.383952    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls" event={"ID":"efb3e082-e7c2-4dad-9104-fefd6d261ad1","Type":"ContainerStarted","Data":"25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3"}
Jul 15 14:24:24.384177 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:24.384009    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls" event={"ID":"efb3e082-e7c2-4dad-9104-fefd6d261ad1","Type":"ContainerStarted","Data":"aa01ac932b69deec33970dd09e88883fa56d6c606858fc0c6c0b179de7c8fdac"}
Jul 15 14:24:24.384177 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:24.384061    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:24.405557 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:24.405501    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls" podStartSLOduration=1.988774754 podStartE2EDuration="2.405483187s" totalImagesPullingTime="416.708433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:24:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:24:24.404809047 +0000 UTC m=+1985.524614630" watchObservedRunningTime="2026-07-15 14:24:24.405483187 +0000 UTC m=+1985.525288800"
Jul 15 14:24:30.395283 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:30.395251    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:24:34.904603 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:34.904559    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"]
Jul 15 14:24:34.908676 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:34.908652    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:34.915358 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:34.915037    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9e73fa92\"/\"default-dockercfg-77m6w\""
Jul 15 14:24:34.915358 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:34.915125    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9e73fa92\"/\"e2e-lifecycle-model-9e73fa92-kserve-self-signed-certs\""
Jul 15 14:24:34.915358 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:34.915192    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9e73fa92\"/\"openshift-service-ca.crt\""
Jul 15 14:24:34.915358 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:34.915237    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9e73fa92\"/\"kube-root-ca.crt\""
Jul 15 14:24:34.921674 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:34.921646    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"]
Jul 15 14:24:35.006158 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.006120    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-home\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.006377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.006235    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-dshm\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.006377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.006279    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4k6g\" (UniqueName: \"kubernetes.io/projected/818039b6-e55f-4860-aa93-c6bd735bbfe9-kube-api-access-k4k6g\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.006377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.006304    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-model-cache\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.006377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.006326    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818039b6-e55f-4860-aa93-c6bd735bbfe9-tls-certs\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.006537 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.006397    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-tmp-dir\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.107813 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.107767    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4k6g\" (UniqueName: \"kubernetes.io/projected/818039b6-e55f-4860-aa93-c6bd735bbfe9-kube-api-access-k4k6g\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.107813 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.107817    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-model-cache\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.108064 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.107905    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818039b6-e55f-4860-aa93-c6bd735bbfe9-tls-certs\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.108064 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.107940    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-tmp-dir\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.108064 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.107968    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-home\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.108213 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.108074    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-dshm\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.108279 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.108257    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-model-cache\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.108382 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.108330    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-tmp-dir\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.108446 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.108413    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-home\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.110350 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.110314    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-dshm\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.110651 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.110632    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818039b6-e55f-4860-aa93-c6bd735bbfe9-tls-certs\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.117941 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.117922    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4k6g\" (UniqueName: \"kubernetes.io/projected/818039b6-e55f-4860-aa93-c6bd735bbfe9-kube-api-access-k4k6g\") pod \"e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") " pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.218580 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.218481    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:35.363065 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.363035    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"]
Jul 15 14:24:35.365807 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:24:35.365766    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod818039b6_e55f_4860_aa93_c6bd735bbfe9.slice/crio-3522d177702362e5477f76d2663e042dd5216d2d89f9b09a20bb9c95297ebb82 WatchSource:0}: Error finding container 3522d177702362e5477f76d2663e042dd5216d2d89f9b09a20bb9c95297ebb82: Status 404 returned error can't find the container with id 3522d177702362e5477f76d2663e042dd5216d2d89f9b09a20bb9c95297ebb82
Jul 15 14:24:35.442591 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.442552    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8" event={"ID":"818039b6-e55f-4860-aa93-c6bd735bbfe9","Type":"ContainerStarted","Data":"a4b3204ed7245c495eefe086225867142f7b96d752b9b1f319525acc0af198dd"}
Jul 15 14:24:35.442778 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.442604    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8" event={"ID":"818039b6-e55f-4860-aa93-c6bd735bbfe9","Type":"ContainerStarted","Data":"3522d177702362e5477f76d2663e042dd5216d2d89f9b09a20bb9c95297ebb82"}
Jul 15 14:24:35.468147 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:35.468094    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8" podStartSLOduration=1.468075364 podStartE2EDuration="1.468075364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:24:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:24:35.46591366 +0000 UTC m=+1996.585719241" watchObservedRunningTime="2026-07-15 14:24:35.468075364 +0000 UTC m=+1996.587880944"
Jul 15 14:24:45.218636 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:45.218598    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:45.219048 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:45.218652    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:45.231251 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:45.231215    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:24:45.506010 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:24:45.505924    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:25:19.526266 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:19.526231    2573 scope.go:117] "RemoveContainer" containerID="d8805b57f1817b082e31149ebbdfd0445429f40b2936abe957dafafdd919ae4b"
Jul 15 14:25:21.581149 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.581108    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"]
Jul 15 14:25:21.581629 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.581316    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls" podUID="efb3e082-e7c2-4dad-9104-fefd6d261ad1" containerName="maas-api" containerID="cri-o://25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3" gracePeriod=30
Jul 15 14:25:21.837119 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.837030    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:25:21.956843 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.956801    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls\") pod \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") "
Jul 15 14:25:21.957047 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.956888    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb3e082-e7c2-4dad-9104-fefd6d261ad1-openshift-service-ca\") pod \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") "
Jul 15 14:25:21.957047 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.956977    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q8qk\" (UniqueName: \"kubernetes.io/projected/efb3e082-e7c2-4dad-9104-fefd6d261ad1-kube-api-access-2q8qk\") pod \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\" (UID: \"efb3e082-e7c2-4dad-9104-fefd6d261ad1\") "
Jul 15 14:25:21.957259 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.957231    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efb3e082-e7c2-4dad-9104-fefd6d261ad1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efb3e082-e7c2-4dad-9104-fefd6d261ad1" (UID: "efb3e082-e7c2-4dad-9104-fefd6d261ad1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:25:21.958951 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.958921    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efb3e082-e7c2-4dad-9104-fefd6d261ad1" (UID: "efb3e082-e7c2-4dad-9104-fefd6d261ad1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:25:21.959060 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:21.959031    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb3e082-e7c2-4dad-9104-fefd6d261ad1-kube-api-access-2q8qk" (OuterVolumeSpecName: "kube-api-access-2q8qk") pod "efb3e082-e7c2-4dad-9104-fefd6d261ad1" (UID: "efb3e082-e7c2-4dad-9104-fefd6d261ad1"). InnerVolumeSpecName "kube-api-access-2q8qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:25:22.058377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.058322    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb3e082-e7c2-4dad-9104-fefd6d261ad1-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:22.058377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.058372    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q8qk\" (UniqueName: \"kubernetes.io/projected/efb3e082-e7c2-4dad-9104-fefd6d261ad1-kube-api-access-2q8qk\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:22.058377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.058382    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb3e082-e7c2-4dad-9104-fefd6d261ad1-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:22.689814 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.689777    2573 generic.go:356] "Generic (PLEG): container finished" podID="efb3e082-e7c2-4dad-9104-fefd6d261ad1" containerID="25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3" exitCode=0
Jul 15 14:25:22.690321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.689823    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"
Jul 15 14:25:22.690321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.689870    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls" event={"ID":"efb3e082-e7c2-4dad-9104-fefd6d261ad1","Type":"ContainerDied","Data":"25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3"}
Jul 15 14:25:22.690321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.689935    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls" event={"ID":"efb3e082-e7c2-4dad-9104-fefd6d261ad1","Type":"ContainerDied","Data":"aa01ac932b69deec33970dd09e88883fa56d6c606858fc0c6c0b179de7c8fdac"}
Jul 15 14:25:22.690321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.689973    2573 scope.go:117] "RemoveContainer" containerID="25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3"
Jul 15 14:25:22.699472 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.699446    2573 scope.go:117] "RemoveContainer" containerID="25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3"
Jul 15 14:25:22.699737 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:25:22.699717    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3\": container with ID starting with 25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3 not found: ID does not exist" containerID="25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3"
Jul 15 14:25:22.699797 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.699748    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3"} err="failed to get container status \"25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3\": rpc error: code = NotFound desc = could not find container \"25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3\": container with ID starting with 25fb5053150fae87a7364eefe572e3471abcd688601eb97fcee74027639667f3 not found: ID does not exist"
Jul 15 14:25:22.716155 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.716123    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"]
Jul 15 14:25:22.719403 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:22.719376    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e73fa92-6b9fbc98c6-df4ls"]
Jul 15 14:25:23.408686 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:23.408647    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efb3e082-e7c2-4dad-9104-fefd6d261ad1" path="/var/lib/kubelet/pods/efb3e082-e7c2-4dad-9104-fefd6d261ad1/volumes"
Jul 15 14:25:30.605541 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:30.605492    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"]
Jul 15 14:25:45.629510 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.629411    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8" podUID="818039b6-e55f-4860-aa93-c6bd735bbfe9" containerName="main" containerID="cri-o://a4b3204ed7245c495eefe086225867142f7b96d752b9b1f319525acc0af198dd" gracePeriod=45
Jul 15 14:25:45.809254 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.809220    2573 generic.go:356] "Generic (PLEG): container finished" podID="818039b6-e55f-4860-aa93-c6bd735bbfe9" containerID="a4b3204ed7245c495eefe086225867142f7b96d752b9b1f319525acc0af198dd" exitCode=0
Jul 15 14:25:45.809427 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.809264    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8" event={"ID":"818039b6-e55f-4860-aa93-c6bd735bbfe9","Type":"ContainerDied","Data":"a4b3204ed7245c495eefe086225867142f7b96d752b9b1f319525acc0af198dd"}
Jul 15 14:25:45.890492 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.890415    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:25:45.983705 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.983668    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-home\") pod \"818039b6-e55f-4860-aa93-c6bd735bbfe9\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") "
Jul 15 14:25:45.983900 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.983715    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4k6g\" (UniqueName: \"kubernetes.io/projected/818039b6-e55f-4860-aa93-c6bd735bbfe9-kube-api-access-k4k6g\") pod \"818039b6-e55f-4860-aa93-c6bd735bbfe9\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") "
Jul 15 14:25:45.983900 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.983751    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-dshm\") pod \"818039b6-e55f-4860-aa93-c6bd735bbfe9\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") "
Jul 15 14:25:45.983900 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.983778    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818039b6-e55f-4860-aa93-c6bd735bbfe9-tls-certs\") pod \"818039b6-e55f-4860-aa93-c6bd735bbfe9\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") "
Jul 15 14:25:45.983900 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.983824    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-tmp-dir\") pod \"818039b6-e55f-4860-aa93-c6bd735bbfe9\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") "
Jul 15 14:25:45.983900 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.983881    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-model-cache\") pod \"818039b6-e55f-4860-aa93-c6bd735bbfe9\" (UID: \"818039b6-e55f-4860-aa93-c6bd735bbfe9\") "
Jul 15 14:25:45.984179 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.983965    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-home" (OuterVolumeSpecName: "home") pod "818039b6-e55f-4860-aa93-c6bd735bbfe9" (UID: "818039b6-e55f-4860-aa93-c6bd735bbfe9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:25:45.984179 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.984140    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "818039b6-e55f-4860-aa93-c6bd735bbfe9" (UID: "818039b6-e55f-4860-aa93-c6bd735bbfe9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:25:45.984179 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.984170    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-model-cache" (OuterVolumeSpecName: "model-cache") pod "818039b6-e55f-4860-aa93-c6bd735bbfe9" (UID: "818039b6-e55f-4860-aa93-c6bd735bbfe9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:25:45.984360 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.984226    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-home\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:45.984360 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.984257    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-tmp-dir\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:45.986101 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.986056    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-dshm" (OuterVolumeSpecName: "dshm") pod "818039b6-e55f-4860-aa93-c6bd735bbfe9" (UID: "818039b6-e55f-4860-aa93-c6bd735bbfe9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:25:45.986215 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.986111    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/818039b6-e55f-4860-aa93-c6bd735bbfe9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "818039b6-e55f-4860-aa93-c6bd735bbfe9" (UID: "818039b6-e55f-4860-aa93-c6bd735bbfe9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:25:45.986215 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:45.986182    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818039b6-e55f-4860-aa93-c6bd735bbfe9-kube-api-access-k4k6g" (OuterVolumeSpecName: "kube-api-access-k4k6g") pod "818039b6-e55f-4860-aa93-c6bd735bbfe9" (UID: "818039b6-e55f-4860-aa93-c6bd735bbfe9"). InnerVolumeSpecName "kube-api-access-k4k6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:25:46.084933 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.084893    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-model-cache\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:46.084933 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.084927    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4k6g\" (UniqueName: \"kubernetes.io/projected/818039b6-e55f-4860-aa93-c6bd735bbfe9-kube-api-access-k4k6g\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:46.084933 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.084940    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818039b6-e55f-4860-aa93-c6bd735bbfe9-dshm\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:46.085174 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.084952    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818039b6-e55f-4860-aa93-c6bd735bbfe9-tls-certs\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:25:46.816128 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.816093    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"
Jul 15 14:25:46.816128 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.816108    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8" event={"ID":"818039b6-e55f-4860-aa93-c6bd735bbfe9","Type":"ContainerDied","Data":"3522d177702362e5477f76d2663e042dd5216d2d89f9b09a20bb9c95297ebb82"}
Jul 15 14:25:46.816623 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.816154    2573 scope.go:117] "RemoveContainer" containerID="a4b3204ed7245c495eefe086225867142f7b96d752b9b1f319525acc0af198dd"
Jul 15 14:25:46.849760 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.849705    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"]
Jul 15 14:25:46.862359 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:46.862313    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-9e73fa92/e2e-lifecycle-model-9e73fa92-kserve-7986c77c4c-gs9q8"]
Jul 15 14:25:47.406713 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:25:47.406666    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818039b6-e55f-4860-aa93-c6bd735bbfe9" path="/var/lib/kubelet/pods/818039b6-e55f-4860-aa93-c6bd735bbfe9/volumes"
Jul 15 14:26:19.516466 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:26:19.516429    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:26:19.519180 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:26:19.519154    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:27:18.382361 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.382262    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"]
Jul 15 14:27:18.382834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.382702    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818039b6-e55f-4860-aa93-c6bd735bbfe9" containerName="main"
Jul 15 14:27:18.382834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.382715    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="818039b6-e55f-4860-aa93-c6bd735bbfe9" containerName="main"
Jul 15 14:27:18.382834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.382727    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb3e082-e7c2-4dad-9104-fefd6d261ad1" containerName="maas-api"
Jul 15 14:27:18.382834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.382732    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb3e082-e7c2-4dad-9104-fefd6d261ad1" containerName="maas-api"
Jul 15 14:27:18.382834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.382805    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="818039b6-e55f-4860-aa93-c6bd735bbfe9" containerName="main"
Jul 15 14:27:18.382834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.382812    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb3e082-e7c2-4dad-9104-fefd6d261ad1" containerName="maas-api"
Jul 15 14:27:18.385969 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.385948    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:18.393113 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.393090    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:27:18.393977 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.393957    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-776ac1-serving-cert\""
Jul 15 14:27:18.394105 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.394002    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:27:18.394105 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.394082    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:27:18.404373 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.404326    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"]
Jul 15 14:27:18.477834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.477792    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:18.478031 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.477853    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wngts\" (UniqueName: \"kubernetes.io/projected/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-kube-api-access-wngts\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:18.478031 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.477931    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:18.578603 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.578550    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngts\" (UniqueName: \"kubernetes.io/projected/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-kube-api-access-wngts\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:18.578603 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.578616    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:18.578848 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.578699    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:18.578848 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:27:18.578797    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-serving-cert: secret "maas-api-e2e-inf-a-776ac1-serving-cert" not found
Jul 15 14:27:18.578925 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:27:18.578863    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls podName:bc75d0c6-6194-441b-85eb-d71d3fa2f8a0 nodeName:}" failed. No retries permitted until 2026-07-15 14:27:19.078844413 +0000 UTC m=+2160.198649976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls") pod "maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c" (UID: "bc75d0c6-6194-441b-85eb-d71d3fa2f8a0") : secret "maas-api-e2e-inf-a-776ac1-serving-cert" not found
Jul 15 14:27:18.579382 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.579363    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:18.600017 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:18.599977    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wngts\" (UniqueName: \"kubernetes.io/projected/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-kube-api-access-wngts\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:19.083390 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:19.083325    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:19.083586 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:27:19.083483    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-serving-cert: secret "maas-api-e2e-inf-a-776ac1-serving-cert" not found
Jul 15 14:27:19.083586 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:27:19.083560    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls podName:bc75d0c6-6194-441b-85eb-d71d3fa2f8a0 nodeName:}" failed. No retries permitted until 2026-07-15 14:27:20.083540068 +0000 UTC m=+2161.203345635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls") pod "maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c" (UID: "bc75d0c6-6194-441b-85eb-d71d3fa2f8a0") : secret "maas-api-e2e-inf-a-776ac1-serving-cert" not found
Jul 15 14:27:20.091983 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:20.091941    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:20.094495 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:20.094472    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls\") pod \"maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:20.206419 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:20.206382    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:27:20.214055 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:20.214032    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:20.363389 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:20.363365    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"]
Jul 15 14:27:20.364994 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:27:20.364970    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc75d0c6_6194_441b_85eb_d71d3fa2f8a0.slice/crio-5a6c9ee4dfc06618f4f1eef7255eb63ecd4d3001dc902fa19eaf6161fe13217a WatchSource:0}: Error finding container 5a6c9ee4dfc06618f4f1eef7255eb63ecd4d3001dc902fa19eaf6161fe13217a: Status 404 returned error can't find the container with id 5a6c9ee4dfc06618f4f1eef7255eb63ecd4d3001dc902fa19eaf6161fe13217a
Jul 15 14:27:21.322778 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:21.322735    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c" event={"ID":"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0","Type":"ContainerStarted","Data":"c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88"}
Jul 15 14:27:21.323293 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:21.322797    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c" event={"ID":"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0","Type":"ContainerStarted","Data":"5a6c9ee4dfc06618f4f1eef7255eb63ecd4d3001dc902fa19eaf6161fe13217a"}
Jul 15 14:27:21.323293 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:21.322856    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:21.352678 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:21.352624    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c" podStartSLOduration=2.924790312 podStartE2EDuration="3.352605714s" totalImagesPullingTime="427.815402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:27:21.350889786 +0000 UTC m=+2162.470695379" watchObservedRunningTime="2026-07-15 14:27:21.352605714 +0000 UTC m=+2162.472411351"
Jul 15 14:27:26.512921 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.512876    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"]
Jul 15 14:27:26.516717 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.516690    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:26.522095 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.522062    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-1bc685-serving-cert\""
Jul 15 14:27:26.537286 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.537246    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"]
Jul 15 14:27:26.653545 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.653505    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:26.653731 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.653570    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqfqn\" (UniqueName: \"kubernetes.io/projected/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-kube-api-access-dqfqn\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:26.653731 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.653658    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:26.754894 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.754848    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:26.755099 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.754926    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqfqn\" (UniqueName: \"kubernetes.io/projected/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-kube-api-access-dqfqn\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:26.755099 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.754995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:26.755099 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:27:26.755022    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-serving-cert: secret "maas-api-e2e-inf-b-1bc685-serving-cert" not found
Jul 15 14:27:26.755272 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:27:26.755103    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls podName:7207e5ae-497b-47ae-a0b1-14b8a6e57cd6 nodeName:}" failed. No retries permitted until 2026-07-15 14:27:27.255081167 +0000 UTC m=+2168.374886734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls") pod "maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk" (UID: "7207e5ae-497b-47ae-a0b1-14b8a6e57cd6") : secret "maas-api-e2e-inf-b-1bc685-serving-cert" not found
Jul 15 14:27:26.755689 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.755668    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:26.769091 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:26.769025    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqfqn\" (UniqueName: \"kubernetes.io/projected/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-kube-api-access-dqfqn\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:27.259470 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:27.259433    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:27.261882 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:27.261848    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls\") pod \"maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:27.333499 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:27.333426    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:27:27.428001 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:27.427967    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:27.572205 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:27.572180    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"]
Jul 15 14:27:27.574704 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:27:27.574677    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7207e5ae_497b_47ae_a0b1_14b8a6e57cd6.slice/crio-6eb40b275c15d4d4cccd2a9df616450e4edfe56f563a17d251ed7e482556e1d3 WatchSource:0}: Error finding container 6eb40b275c15d4d4cccd2a9df616450e4edfe56f563a17d251ed7e482556e1d3: Status 404 returned error can't find the container with id 6eb40b275c15d4d4cccd2a9df616450e4edfe56f563a17d251ed7e482556e1d3
Jul 15 14:27:28.366227 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:28.366178    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk" event={"ID":"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6","Type":"ContainerStarted","Data":"6eb40b275c15d4d4cccd2a9df616450e4edfe56f563a17d251ed7e482556e1d3"}
Jul 15 14:27:29.372348 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:29.372304    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk" event={"ID":"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6","Type":"ContainerStarted","Data":"3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00"}
Jul 15 14:27:29.372766 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:29.372493    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:29.406766 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:29.406706    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk" podStartSLOduration=2.589178543 podStartE2EDuration="3.406684732s" totalImagesPullingTime="817.506189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:27:29.404773943 +0000 UTC m=+2170.524579535" watchObservedRunningTime="2026-07-15 14:27:29.406684732 +0000 UTC m=+2170.526490314"
Jul 15 14:27:35.382980 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:35.382947    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:27:48.898626 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.898585    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"]
Jul 15 14:27:48.902104 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.902071    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:48.906061 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.906029    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-776ac1\"/\"test-model-776ac1-kserve-self-signed-certs\""
Jul 15 14:27:48.906061 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.906050    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-776ac1\"/\"kube-root-ca.crt\""
Jul 15 14:27:48.906256 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.906030    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-776ac1\"/\"openshift-service-ca.crt\""
Jul 15 14:27:48.906256 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.906107    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-776ac1\"/\"default-dockercfg-njndw\""
Jul 15 14:27:48.912640 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.912612    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"]
Jul 15 14:27:48.973277 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.973235    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-dshm\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:48.973477 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.973292    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tls-certs\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:48.973477 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.973354    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-model-cache\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:48.973477 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.973370    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-home\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:48.973477 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.973431    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zg5b\" (UniqueName: \"kubernetes.io/projected/f073cdfb-b822-4fcc-a0f0-417d979d06ce-kube-api-access-2zg5b\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:48.973617 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:48.973491    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tmp-dir\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.074899 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.074856    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-dshm\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.075089 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.074916    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tls-certs\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.075089 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.074943    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-model-cache\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.075089 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.074967    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-home\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.075089 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.075025    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zg5b\" (UniqueName: \"kubernetes.io/projected/f073cdfb-b822-4fcc-a0f0-417d979d06ce-kube-api-access-2zg5b\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.075089 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.075061    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tmp-dir\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.075424 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.075395    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-model-cache\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.075514 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.075424    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-home\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.075514 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.075490    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tmp-dir\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.077216 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.077181    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-dshm\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.077591 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.077570    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tls-certs\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.084321 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.084291    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zg5b\" (UniqueName: \"kubernetes.io/projected/f073cdfb-b822-4fcc-a0f0-417d979d06ce-kube-api-access-2zg5b\") pod \"test-model-776ac1-kserve-584df9c4bc-nz4j9\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") " pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.213106 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.213008    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:49.364907 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.364880    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"]
Jul 15 14:27:49.366794 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:27:49.366765    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf073cdfb_b822_4fcc_a0f0_417d979d06ce.slice/crio-d424dcb79b25c624dcfc2d78d6ad6d99ac9ff45d30d3a351abfccb38ad011d4c WatchSource:0}: Error finding container d424dcb79b25c624dcfc2d78d6ad6d99ac9ff45d30d3a351abfccb38ad011d4c: Status 404 returned error can't find the container with id d424dcb79b25c624dcfc2d78d6ad6d99ac9ff45d30d3a351abfccb38ad011d4c
Jul 15 14:27:49.486770 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.486678    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9" event={"ID":"f073cdfb-b822-4fcc-a0f0-417d979d06ce","Type":"ContainerStarted","Data":"53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1"}
Jul 15 14:27:49.486770 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.486722    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9" event={"ID":"f073cdfb-b822-4fcc-a0f0-417d979d06ce","Type":"ContainerStarted","Data":"d424dcb79b25c624dcfc2d78d6ad6d99ac9ff45d30d3a351abfccb38ad011d4c"}
Jul 15 14:27:49.509582 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:49.509529    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9" podStartSLOduration=1.5095100989999999 podStartE2EDuration="1.509510099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:27:49.507557425 +0000 UTC m=+2190.627363008" watchObservedRunningTime="2026-07-15 14:27:49.509510099 +0000 UTC m=+2190.629315909"
Jul 15 14:27:59.213980 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:59.213937    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:59.214595 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:59.214000    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:59.227197 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:59.227165    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:27:59.551087 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:27:59.551053    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:28:14.099741 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.099697    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"]
Jul 15 14:28:14.104011 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.103981    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.107443 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.107412    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1bc685\"/\"openshift-service-ca.crt\""
Jul 15 14:28:14.108642 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.108621    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1bc685\"/\"default-dockercfg-t9s2k\""
Jul 15 14:28:14.108742 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.108621    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1bc685\"/\"kube-root-ca.crt\""
Jul 15 14:28:14.108742 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.108627    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1bc685\"/\"test-model-1bc685-kserve-self-signed-certs\""
Jul 15 14:28:14.115453 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.115426    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"]
Jul 15 14:28:14.216671 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.216617    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-home\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.216849 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.216684    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrfs2\" (UniqueName: \"kubernetes.io/projected/302b1ba1-4e0e-46cf-b945-f757ec466b55-kube-api-access-jrfs2\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.216849 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.216765    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-dshm\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.216927 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.216876    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-tmp-dir\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.216927 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.216907    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-model-cache\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.216997 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.216931    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302b1ba1-4e0e-46cf-b945-f757ec466b55-tls-certs\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318055 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318015    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-home\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318249 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318064    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrfs2\" (UniqueName: \"kubernetes.io/projected/302b1ba1-4e0e-46cf-b945-f757ec466b55-kube-api-access-jrfs2\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318249 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318104    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-dshm\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318249 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318169    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-tmp-dir\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318249 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318189    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-model-cache\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318461 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318316    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302b1ba1-4e0e-46cf-b945-f757ec466b55-tls-certs\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318539 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318514    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-home\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318604 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318537    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-model-cache\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.318709 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.318691    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-tmp-dir\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.320566 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.320546    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-dshm\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.320850 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.320827    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302b1ba1-4e0e-46cf-b945-f757ec466b55-tls-certs\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.338902 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.338860    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrfs2\" (UniqueName: \"kubernetes.io/projected/302b1ba1-4e0e-46cf-b945-f757ec466b55-kube-api-access-jrfs2\") pod \"test-model-1bc685-kserve-5668b77f79-dc84c\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") " pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.415081 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.414979    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:14.568836 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.568802    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"]
Jul 15 14:28:14.569850 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:28:14.569823    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod302b1ba1_4e0e_46cf_b945_f757ec466b55.slice/crio-516ffe0927ff2a1bc009681bbb86865a728282e1e4f4927ee44907725766bc17 WatchSource:0}: Error finding container 516ffe0927ff2a1bc009681bbb86865a728282e1e4f4927ee44907725766bc17: Status 404 returned error can't find the container with id 516ffe0927ff2a1bc009681bbb86865a728282e1e4f4927ee44907725766bc17
Jul 15 14:28:14.623094 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:14.623062    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c" event={"ID":"302b1ba1-4e0e-46cf-b945-f757ec466b55","Type":"ContainerStarted","Data":"516ffe0927ff2a1bc009681bbb86865a728282e1e4f4927ee44907725766bc17"}
Jul 15 14:28:15.630964 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:15.630927    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c" event={"ID":"302b1ba1-4e0e-46cf-b945-f757ec466b55","Type":"ContainerStarted","Data":"bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0"}
Jul 15 14:28:15.658598 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:15.658544    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c" podStartSLOduration=1.658523913 podStartE2EDuration="1.658523913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:28:15.65652756 +0000 UTC m=+2216.776333142" watchObservedRunningTime="2026-07-15 14:28:15.658523913 +0000 UTC m=+2216.778329553"
Jul 15 14:28:24.415500 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:24.415439    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:24.415500 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:24.415513    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:24.428478 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:24.428446    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:24.688720 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:24.688631    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:40.176747 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:40.176706    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"]
Jul 15 14:28:41.136631 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:41.136541    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"]
Jul 15 14:28:51.814825 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:51.814789    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"]
Jul 15 14:28:51.815430 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:51.815053    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c" podUID="bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" containerName="maas-api" containerID="cri-o://c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88" gracePeriod=30
Jul 15 14:28:52.067270 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.067195    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:28:52.077575 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.077546    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wngts\" (UniqueName: \"kubernetes.io/projected/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-kube-api-access-wngts\") pod \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") "
Jul 15 14:28:52.077704 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.077585    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-openshift-service-ca\") pod \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") "
Jul 15 14:28:52.077704 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.077639    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls\") pod \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\" (UID: \"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0\") "
Jul 15 14:28:52.078017 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.077987    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" (UID: "bc75d0c6-6194-441b-85eb-d71d3fa2f8a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:28:52.079759 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.079733    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" (UID: "bc75d0c6-6194-441b-85eb-d71d3fa2f8a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:28:52.079841 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.079804    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-kube-api-access-wngts" (OuterVolumeSpecName: "kube-api-access-wngts") pod "bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" (UID: "bc75d0c6-6194-441b-85eb-d71d3fa2f8a0"). InnerVolumeSpecName "kube-api-access-wngts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:28:52.178721 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.178667    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wngts\" (UniqueName: \"kubernetes.io/projected/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-kube-api-access-wngts\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:52.178721 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.178714    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:52.178721 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.178731    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:52.830678 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.830634    2573 generic.go:356] "Generic (PLEG): container finished" podID="bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" containerID="c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88" exitCode=0
Jul 15 14:28:52.831081 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.830676    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c" event={"ID":"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0","Type":"ContainerDied","Data":"c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88"}
Jul 15 14:28:52.831081 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.830699    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"
Jul 15 14:28:52.831081 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.830761    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c" event={"ID":"bc75d0c6-6194-441b-85eb-d71d3fa2f8a0","Type":"ContainerDied","Data":"5a6c9ee4dfc06618f4f1eef7255eb63ecd4d3001dc902fa19eaf6161fe13217a"}
Jul 15 14:28:52.831081 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.830785    2573 scope.go:117] "RemoveContainer" containerID="c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88"
Jul 15 14:28:52.843837 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.843812    2573 scope.go:117] "RemoveContainer" containerID="c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88"
Jul 15 14:28:52.844206 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:28:52.844182    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88\": container with ID starting with c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88 not found: ID does not exist" containerID="c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88"
Jul 15 14:28:52.844265 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.844219    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88"} err="failed to get container status \"c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88\": rpc error: code = NotFound desc = could not find container \"c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88\": container with ID starting with c1a70e8f96d3485cc94722ef0dc7328034059abb8ab2f055325e7dc4eb28aa88 not found: ID does not exist"
Jul 15 14:28:52.868136 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.868099    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"]
Jul 15 14:28:52.876904 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:52.876871    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-776ac1-6d89c5fbcc-ljp5c"]
Jul 15 14:28:53.407352 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:53.407310    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" path="/var/lib/kubelet/pods/bc75d0c6-6194-441b-85eb-d71d3fa2f8a0/volumes"
Jul 15 14:28:55.197088 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.197031    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9" podUID="f073cdfb-b822-4fcc-a0f0-417d979d06ce" containerName="main" containerID="cri-o://53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1" gracePeriod=45
Jul 15 14:28:55.461955 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.461887    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:28:55.513732 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.513696    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tmp-dir\") pod \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") "
Jul 15 14:28:55.513922 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.513747    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tls-certs\") pod \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") "
Jul 15 14:28:55.513922 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.513768    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-model-cache\") pod \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") "
Jul 15 14:28:55.513922 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.513893    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-dshm\") pod \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") "
Jul 15 14:28:55.514101 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.513950    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zg5b\" (UniqueName: \"kubernetes.io/projected/f073cdfb-b822-4fcc-a0f0-417d979d06ce-kube-api-access-2zg5b\") pod \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") "
Jul 15 14:28:55.514101 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.513982    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-home\") pod \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\" (UID: \"f073cdfb-b822-4fcc-a0f0-417d979d06ce\") "
Jul 15 14:28:55.514101 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.514021    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f073cdfb-b822-4fcc-a0f0-417d979d06ce" (UID: "f073cdfb-b822-4fcc-a0f0-417d979d06ce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:28:55.514101 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.514038    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-model-cache" (OuterVolumeSpecName: "model-cache") pod "f073cdfb-b822-4fcc-a0f0-417d979d06ce" (UID: "f073cdfb-b822-4fcc-a0f0-417d979d06ce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:28:55.514372 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.514321    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-home" (OuterVolumeSpecName: "home") pod "f073cdfb-b822-4fcc-a0f0-417d979d06ce" (UID: "f073cdfb-b822-4fcc-a0f0-417d979d06ce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:28:55.514439 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.514381    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tmp-dir\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:55.514439 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.514402    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-model-cache\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:55.516165 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.516129    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f073cdfb-b822-4fcc-a0f0-417d979d06ce-kube-api-access-2zg5b" (OuterVolumeSpecName: "kube-api-access-2zg5b") pod "f073cdfb-b822-4fcc-a0f0-417d979d06ce" (UID: "f073cdfb-b822-4fcc-a0f0-417d979d06ce"). InnerVolumeSpecName "kube-api-access-2zg5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:28:55.516454 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.516433    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-dshm" (OuterVolumeSpecName: "dshm") pod "f073cdfb-b822-4fcc-a0f0-417d979d06ce" (UID: "f073cdfb-b822-4fcc-a0f0-417d979d06ce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:28:55.516454 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.516444    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f073cdfb-b822-4fcc-a0f0-417d979d06ce" (UID: "f073cdfb-b822-4fcc-a0f0-417d979d06ce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:28:55.615633 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.615592    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-home\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:55.615633 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.615626    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f073cdfb-b822-4fcc-a0f0-417d979d06ce-tls-certs\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:55.615633 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.615635    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f073cdfb-b822-4fcc-a0f0-417d979d06ce-dshm\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:55.615633 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.615643    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zg5b\" (UniqueName: \"kubernetes.io/projected/f073cdfb-b822-4fcc-a0f0-417d979d06ce-kube-api-access-2zg5b\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:55.849006 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.848966    2573 generic.go:356] "Generic (PLEG): container finished" podID="f073cdfb-b822-4fcc-a0f0-417d979d06ce" containerID="53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1" exitCode=0
Jul 15 14:28:55.849200 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.849012    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"
Jul 15 14:28:55.849200 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.849051    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9" event={"ID":"f073cdfb-b822-4fcc-a0f0-417d979d06ce","Type":"ContainerDied","Data":"53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1"}
Jul 15 14:28:55.849200 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.849094    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9" event={"ID":"f073cdfb-b822-4fcc-a0f0-417d979d06ce","Type":"ContainerDied","Data":"d424dcb79b25c624dcfc2d78d6ad6d99ac9ff45d30d3a351abfccb38ad011d4c"}
Jul 15 14:28:55.849200 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.849115    2573 scope.go:117] "RemoveContainer" containerID="53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1"
Jul 15 14:28:55.858668 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.858650    2573 scope.go:117] "RemoveContainer" containerID="53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1"
Jul 15 14:28:55.858972 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:28:55.858951    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1\": container with ID starting with 53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1 not found: ID does not exist" containerID="53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1"
Jul 15 14:28:55.859023 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.858983    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1"} err="failed to get container status \"53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1\": rpc error: code = NotFound desc = could not find container \"53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1\": container with ID starting with 53d84cac1551edb91e08da54077e1b87d516a29d8e54dc41abc456736ad70bb1 not found: ID does not exist"
Jul 15 14:28:55.873133 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.873097    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"]
Jul 15 14:28:55.877048 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:55.877017    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-776ac1/test-model-776ac1-kserve-584df9c4bc-nz4j9"]
Jul 15 14:28:56.157641 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.157542    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c" podUID="302b1ba1-4e0e-46cf-b945-f757ec466b55" containerName="main" containerID="cri-o://bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0" gracePeriod=45
Jul 15 14:28:56.433020 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.432987    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:56.524651 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.524613    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-dshm\") pod \"302b1ba1-4e0e-46cf-b945-f757ec466b55\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") "
Jul 15 14:28:56.524866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.524677    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-tmp-dir\") pod \"302b1ba1-4e0e-46cf-b945-f757ec466b55\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") "
Jul 15 14:28:56.524866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.524726    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-model-cache\") pod \"302b1ba1-4e0e-46cf-b945-f757ec466b55\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") "
Jul 15 14:28:56.524866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.524769    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrfs2\" (UniqueName: \"kubernetes.io/projected/302b1ba1-4e0e-46cf-b945-f757ec466b55-kube-api-access-jrfs2\") pod \"302b1ba1-4e0e-46cf-b945-f757ec466b55\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") "
Jul 15 14:28:56.524866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.524795    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302b1ba1-4e0e-46cf-b945-f757ec466b55-tls-certs\") pod \"302b1ba1-4e0e-46cf-b945-f757ec466b55\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") "
Jul 15 14:28:56.524866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.524827    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-home\") pod \"302b1ba1-4e0e-46cf-b945-f757ec466b55\" (UID: \"302b1ba1-4e0e-46cf-b945-f757ec466b55\") "
Jul 15 14:28:56.525117 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.524954    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-model-cache" (OuterVolumeSpecName: "model-cache") pod "302b1ba1-4e0e-46cf-b945-f757ec466b55" (UID: "302b1ba1-4e0e-46cf-b945-f757ec466b55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:28:56.525117 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.524962    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "302b1ba1-4e0e-46cf-b945-f757ec466b55" (UID: "302b1ba1-4e0e-46cf-b945-f757ec466b55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:28:56.525278 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.525148    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-home" (OuterVolumeSpecName: "home") pod "302b1ba1-4e0e-46cf-b945-f757ec466b55" (UID: "302b1ba1-4e0e-46cf-b945-f757ec466b55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:28:56.525278 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.525231    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-model-cache\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:56.525278 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.525245    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-home\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:56.525278 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.525258    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-tmp-dir\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:56.526992 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.526957    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-dshm" (OuterVolumeSpecName: "dshm") pod "302b1ba1-4e0e-46cf-b945-f757ec466b55" (UID: "302b1ba1-4e0e-46cf-b945-f757ec466b55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:28:56.527116 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.527020    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/302b1ba1-4e0e-46cf-b945-f757ec466b55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "302b1ba1-4e0e-46cf-b945-f757ec466b55" (UID: "302b1ba1-4e0e-46cf-b945-f757ec466b55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:28:56.527116 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.527015    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/302b1ba1-4e0e-46cf-b945-f757ec466b55-kube-api-access-jrfs2" (OuterVolumeSpecName: "kube-api-access-jrfs2") pod "302b1ba1-4e0e-46cf-b945-f757ec466b55" (UID: "302b1ba1-4e0e-46cf-b945-f757ec466b55"). InnerVolumeSpecName "kube-api-access-jrfs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:28:56.626266 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.626217    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrfs2\" (UniqueName: \"kubernetes.io/projected/302b1ba1-4e0e-46cf-b945-f757ec466b55-kube-api-access-jrfs2\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:56.626266 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.626256    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302b1ba1-4e0e-46cf-b945-f757ec466b55-tls-certs\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:56.626266 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.626268    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302b1ba1-4e0e-46cf-b945-f757ec466b55-dshm\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:28:56.856254 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.856213    2573 generic.go:356] "Generic (PLEG): container finished" podID="302b1ba1-4e0e-46cf-b945-f757ec466b55" containerID="bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0" exitCode=0
Jul 15 14:28:56.856254 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.856255    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"
Jul 15 14:28:56.856520 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.856302    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c" event={"ID":"302b1ba1-4e0e-46cf-b945-f757ec466b55","Type":"ContainerDied","Data":"bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0"}
Jul 15 14:28:56.856520 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.856366    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c" event={"ID":"302b1ba1-4e0e-46cf-b945-f757ec466b55","Type":"ContainerDied","Data":"516ffe0927ff2a1bc009681bbb86865a728282e1e4f4927ee44907725766bc17"}
Jul 15 14:28:56.856520 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.856409    2573 scope.go:117] "RemoveContainer" containerID="bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0"
Jul 15 14:28:56.865790 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.865769    2573 scope.go:117] "RemoveContainer" containerID="bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0"
Jul 15 14:28:56.866147 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:28:56.866127    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0\": container with ID starting with bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0 not found: ID does not exist" containerID="bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0"
Jul 15 14:28:56.866194 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.866156    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0"} err="failed to get container status \"bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0\": rpc error: code = NotFound desc = could not find container \"bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0\": container with ID starting with bbacdad89449ed31d8cb16156ef4b8dcf5643fb571f2ec639afabbc2a2a30eb0 not found: ID does not exist"
Jul 15 14:28:56.880395 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.880355    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"]
Jul 15 14:28:56.887082 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:56.887053    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1bc685/test-model-1bc685-kserve-5668b77f79-dc84c"]
Jul 15 14:28:57.406743 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:57.406707    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="302b1ba1-4e0e-46cf-b945-f757ec466b55" path="/var/lib/kubelet/pods/302b1ba1-4e0e-46cf-b945-f757ec466b55/volumes"
Jul 15 14:28:57.407115 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:28:57.407101    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f073cdfb-b822-4fcc-a0f0-417d979d06ce" path="/var/lib/kubelet/pods/f073cdfb-b822-4fcc-a0f0-417d979d06ce/volumes"
Jul 15 14:29:07.732924 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.732887    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"]
Jul 15 14:29:07.733315 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733289    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f073cdfb-b822-4fcc-a0f0-417d979d06ce" containerName="main"
Jul 15 14:29:07.733315 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733300    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f073cdfb-b822-4fcc-a0f0-417d979d06ce" containerName="main"
Jul 15 14:29:07.733427 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733353    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="302b1ba1-4e0e-46cf-b945-f757ec466b55" containerName="main"
Jul 15 14:29:07.733427 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733359    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="302b1ba1-4e0e-46cf-b945-f757ec466b55" containerName="main"
Jul 15 14:29:07.733427 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733368    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" containerName="maas-api"
Jul 15 14:29:07.733427 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733373    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" containerName="maas-api"
Jul 15 14:29:07.733551 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733428    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc75d0c6-6194-441b-85eb-d71d3fa2f8a0" containerName="maas-api"
Jul 15 14:29:07.733551 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733437    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f073cdfb-b822-4fcc-a0f0-417d979d06ce" containerName="main"
Jul 15 14:29:07.733551 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.733448    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="302b1ba1-4e0e-46cf-b945-f757ec466b55" containerName="main"
Jul 15 14:29:07.737960 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.737938    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"
Jul 15 14:29:07.747936 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.747908    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\""
Jul 15 14:29:07.767620 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.767580    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"]
Jul 15 14:29:07.833689 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.833636    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceba5b1c-3734-4148-a96b-806b6fc9328b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc\" (UID: \"ceba5b1c-3734-4148-a96b-806b6fc9328b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"
Jul 15 14:29:07.935084 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.935046    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceba5b1c-3734-4148-a96b-806b6fc9328b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc\" (UID: \"ceba5b1c-3734-4148-a96b-806b6fc9328b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"
Jul 15 14:29:07.935764 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:07.935744    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceba5b1c-3734-4148-a96b-806b6fc9328b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc\" (UID: \"ceba5b1c-3734-4148-a96b-806b6fc9328b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"
Jul 15 14:29:08.048674 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:08.048638    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"
Jul 15 14:29:08.395497 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:08.395462    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"]
Jul 15 14:29:08.396116 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:29:08.396086    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceba5b1c_3734_4148_a96b_806b6fc9328b.slice/crio-a2f74c604f5073113476bdc3f58762f2881bad326860dc973298ad2f709402bc WatchSource:0}: Error finding container a2f74c604f5073113476bdc3f58762f2881bad326860dc973298ad2f709402bc: Status 404 returned error can't find the container with id a2f74c604f5073113476bdc3f58762f2881bad326860dc973298ad2f709402bc
Jul 15 14:29:08.923014 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:08.922973    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc" event={"ID":"ceba5b1c-3734-4148-a96b-806b6fc9328b","Type":"ContainerStarted","Data":"ae685826302f28894a3a5f77c6345632c21aa0978817e089e0afc05acb8d3271"}
Jul 15 14:29:08.923470 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:08.923034    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc" event={"ID":"ceba5b1c-3734-4148-a96b-806b6fc9328b","Type":"ContainerStarted","Data":"a2f74c604f5073113476bdc3f58762f2881bad326860dc973298ad2f709402bc"}
Jul 15 14:29:08.944646 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:08.944589    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc" podStartSLOduration=1.944573482 podStartE2EDuration="1.944573482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:29:08.941572172 +0000 UTC m=+2270.061377754" watchObservedRunningTime="2026-07-15 14:29:08.944573482 +0000 UTC m=+2270.064379062"
Jul 15 14:29:09.929970 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:09.929935    2573 generic.go:356] "Generic (PLEG): container finished" podID="ceba5b1c-3734-4148-a96b-806b6fc9328b" containerID="ae685826302f28894a3a5f77c6345632c21aa0978817e089e0afc05acb8d3271" exitCode=0
Jul 15 14:29:09.929970 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:09.929975    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc" event={"ID":"ceba5b1c-3734-4148-a96b-806b6fc9328b","Type":"ContainerDied","Data":"ae685826302f28894a3a5f77c6345632c21aa0978817e089e0afc05acb8d3271"}
Jul 15 14:29:11.078546 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:11.078511    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"
Jul 15 14:29:11.169038 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:11.168997    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceba5b1c-3734-4148-a96b-806b6fc9328b-maas-api-service-ca\") pod \"ceba5b1c-3734-4148-a96b-806b6fc9328b\" (UID: \"ceba5b1c-3734-4148-a96b-806b6fc9328b\") "
Jul 15 14:29:11.169430 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:11.169405    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceba5b1c-3734-4148-a96b-806b6fc9328b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ceba5b1c-3734-4148-a96b-806b6fc9328b" (UID: "ceba5b1c-3734-4148-a96b-806b6fc9328b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:29:11.270330 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:11.270252    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceba5b1c-3734-4148-a96b-806b6fc9328b-maas-api-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:29:11.942589 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:11.942556    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc"
Jul 15 14:29:11.942770 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:11.942598    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc" event={"ID":"ceba5b1c-3734-4148-a96b-806b6fc9328b","Type":"ContainerDied","Data":"a2f74c604f5073113476bdc3f58762f2881bad326860dc973298ad2f709402bc"}
Jul 15 14:29:11.942770 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:11.942682    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2f74c604f5073113476bdc3f58762f2881bad326860dc973298ad2f709402bc"
Jul 15 14:29:18.481516 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.481475    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"]
Jul 15 14:29:18.481939 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.481696    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk" podUID="7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" containerName="maas-api" containerID="cri-o://3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00" gracePeriod=30
Jul 15 14:29:18.744582 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.744499    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:29:18.844015 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.843973    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-openshift-service-ca\") pod \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") "
Jul 15 14:29:18.844219 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.844071    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqfqn\" (UniqueName: \"kubernetes.io/projected/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-kube-api-access-dqfqn\") pod \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") "
Jul 15 14:29:18.844219 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.844123    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls\") pod \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\" (UID: \"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6\") "
Jul 15 14:29:18.844415 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.844388    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" (UID: "7207e5ae-497b-47ae-a0b1-14b8a6e57cd6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:29:18.846267 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.846237    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-kube-api-access-dqfqn" (OuterVolumeSpecName: "kube-api-access-dqfqn") pod "7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" (UID: "7207e5ae-497b-47ae-a0b1-14b8a6e57cd6"). InnerVolumeSpecName "kube-api-access-dqfqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:29:18.846363 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.846245    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" (UID: "7207e5ae-497b-47ae-a0b1-14b8a6e57cd6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:29:18.945152 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.945107    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqfqn\" (UniqueName: \"kubernetes.io/projected/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-kube-api-access-dqfqn\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:29:18.945152 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.945145    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:29:18.945152 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.945155    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:29:18.984013 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.983979    2573 generic.go:356] "Generic (PLEG): container finished" podID="7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" containerID="3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00" exitCode=0
Jul 15 14:29:18.984206 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.984026    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"
Jul 15 14:29:18.984206 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.984063    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk" event={"ID":"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6","Type":"ContainerDied","Data":"3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00"}
Jul 15 14:29:18.984206 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.984128    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk" event={"ID":"7207e5ae-497b-47ae-a0b1-14b8a6e57cd6","Type":"ContainerDied","Data":"6eb40b275c15d4d4cccd2a9df616450e4edfe56f563a17d251ed7e482556e1d3"}
Jul 15 14:29:18.984206 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.984152    2573 scope.go:117] "RemoveContainer" containerID="3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00"
Jul 15 14:29:18.993855 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.993826    2573 scope.go:117] "RemoveContainer" containerID="3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00"
Jul 15 14:29:18.994188 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:29:18.994166    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00\": container with ID starting with 3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00 not found: ID does not exist" containerID="3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00"
Jul 15 14:29:18.994251 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:18.994198    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00"} err="failed to get container status \"3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00\": rpc error: code = NotFound desc = could not find container \"3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00\": container with ID starting with 3ecce180b5a6d03a82aa1234d3170af1ecac02cf262394a3d2816df029c48b00 not found: ID does not exist"
Jul 15 14:29:19.009326 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:19.009225    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"]
Jul 15 14:29:19.013492 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:19.013459    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1bc685-6d47c5d84-sctmk"]
Jul 15 14:29:19.407378 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:19.407324    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" path="/var/lib/kubelet/pods/7207e5ae-497b-47ae-a0b1-14b8a6e57cd6/volumes"
Jul 15 14:29:38.152277 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.152238    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"]
Jul 15 14:29:38.152877 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.152678    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" containerName="maas-api"
Jul 15 14:29:38.152877 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.152691    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" containerName="maas-api"
Jul 15 14:29:38.152877 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.152718    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceba5b1c-3734-4148-a96b-806b6fc9328b" containerName="revoke-keys"
Jul 15 14:29:38.152877 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.152726    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceba5b1c-3734-4148-a96b-806b6fc9328b" containerName="revoke-keys"
Jul 15 14:29:38.152877 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.152794    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceba5b1c-3734-4148-a96b-806b6fc9328b" containerName="revoke-keys"
Jul 15 14:29:38.152877 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.152808    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7207e5ae-497b-47ae-a0b1-14b8a6e57cd6" containerName="maas-api"
Jul 15 14:29:38.156178 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.156152    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.166795 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.166769    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-752a1c-serving-cert\""
Jul 15 14:29:38.166949 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.166895    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:29:38.167874 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.167855    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:29:38.167966 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.167862    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:29:38.177192 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.177159    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"]
Jul 15 14:29:38.228231 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.228184    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.228461 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.228238    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbf31ec-ad92-4301-b517-b13fd01215ce-openshift-service-ca\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.228461 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.228275    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2l6f\" (UniqueName: \"kubernetes.io/projected/edbf31ec-ad92-4301-b517-b13fd01215ce-kube-api-access-m2l6f\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.329041 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.328994    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.329041 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.329042    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbf31ec-ad92-4301-b517-b13fd01215ce-openshift-service-ca\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.329269 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:29:38.329152    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-serving-cert: secret "maas-api-e2e-api-a-752a1c-serving-cert" not found
Jul 15 14:29:38.329269 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.329162    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2l6f\" (UniqueName: \"kubernetes.io/projected/edbf31ec-ad92-4301-b517-b13fd01215ce-kube-api-access-m2l6f\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.329269 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:29:38.329227    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls podName:edbf31ec-ad92-4301-b517-b13fd01215ce nodeName:}" failed. No retries permitted until 2026-07-15 14:29:38.829209985 +0000 UTC m=+2299.949015544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls") pod "maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg" (UID: "edbf31ec-ad92-4301-b517-b13fd01215ce") : secret "maas-api-e2e-api-a-752a1c-serving-cert" not found
Jul 15 14:29:38.329704 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.329687    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbf31ec-ad92-4301-b517-b13fd01215ce-openshift-service-ca\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.343875 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.343843    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2l6f\" (UniqueName: \"kubernetes.io/projected/edbf31ec-ad92-4301-b517-b13fd01215ce-kube-api-access-m2l6f\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.833290 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.833245    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:38.835795 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:38.835758    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls\") pod \"maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:39.067029 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:39.066992    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:39.223648 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:39.223624    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"]
Jul 15 14:29:39.226053 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:29:39.226026    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedbf31ec_ad92_4301_b517_b13fd01215ce.slice/crio-0381168f650c7079e2fa16560dc54599b080674f77fe4f0572a7644aa8f7acde WatchSource:0}: Error finding container 0381168f650c7079e2fa16560dc54599b080674f77fe4f0572a7644aa8f7acde: Status 404 returned error can't find the container with id 0381168f650c7079e2fa16560dc54599b080674f77fe4f0572a7644aa8f7acde
Jul 15 14:29:39.227812 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:39.227793    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 14:29:40.100433 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:40.100387    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg" event={"ID":"edbf31ec-ad92-4301-b517-b13fd01215ce","Type":"ContainerStarted","Data":"445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b"}
Jul 15 14:29:40.100644 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:40.100449    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg" event={"ID":"edbf31ec-ad92-4301-b517-b13fd01215ce","Type":"ContainerStarted","Data":"0381168f650c7079e2fa16560dc54599b080674f77fe4f0572a7644aa8f7acde"}
Jul 15 14:29:40.100644 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:40.100469    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:40.136751 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:40.136697    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg" podStartSLOduration=1.62047988 podStartE2EDuration="2.136678599s" totalImagesPullingTime="516.198719ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:29:40.134800227 +0000 UTC m=+2301.254605809" watchObservedRunningTime="2026-07-15 14:29:40.136678599 +0000 UTC m=+2301.256484180"
Jul 15 14:29:46.111273 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.111242    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:29:46.362890 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.362787    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"]
Jul 15 14:29:46.366868 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.366842    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:46.378567 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.378544    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e7e87a-serving-cert\""
Jul 15 14:29:46.385746 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.385717    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"]
Jul 15 14:29:46.409025 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.408985    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:46.409216 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.409070    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwjfc\" (UniqueName: \"kubernetes.io/projected/cc26eca5-9e87-4ec1-b871-094342f5f1da-kube-api-access-fwjfc\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:46.409216 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.409111    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc26eca5-9e87-4ec1-b871-094342f5f1da-openshift-service-ca\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:46.510145 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.510108    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:46.510377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.510155    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwjfc\" (UniqueName: \"kubernetes.io/projected/cc26eca5-9e87-4ec1-b871-094342f5f1da-kube-api-access-fwjfc\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:46.510377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.510178    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc26eca5-9e87-4ec1-b871-094342f5f1da-openshift-service-ca\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:46.510377 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:29:46.510291    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-serving-cert: secret "maas-api-e2e-api-b-e7e87a-serving-cert" not found
Jul 15 14:29:46.510568 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:29:46.510409    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls podName:cc26eca5-9e87-4ec1-b871-094342f5f1da nodeName:}" failed. No retries permitted until 2026-07-15 14:29:47.010387055 +0000 UTC m=+2308.130192616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls") pod "maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw" (UID: "cc26eca5-9e87-4ec1-b871-094342f5f1da") : secret "maas-api-e2e-api-b-e7e87a-serving-cert" not found
Jul 15 14:29:46.510905 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.510883    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc26eca5-9e87-4ec1-b871-094342f5f1da-openshift-service-ca\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:46.528243 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:46.528208    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwjfc\" (UniqueName: \"kubernetes.io/projected/cc26eca5-9e87-4ec1-b871-094342f5f1da-kube-api-access-fwjfc\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:47.014692 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:47.014652    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:47.017156 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:47.017125    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls\") pod \"maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:47.277393 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:47.277358    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:47.444821 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:47.444739    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"]
Jul 15 14:29:47.447262 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:29:47.447230    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc26eca5_9e87_4ec1_b871_094342f5f1da.slice/crio-2e437690ee7003c0f67360f0ab8d09d7ccf5360bd2fb3a7d6b9ee33c0894301b WatchSource:0}: Error finding container 2e437690ee7003c0f67360f0ab8d09d7ccf5360bd2fb3a7d6b9ee33c0894301b: Status 404 returned error can't find the container with id 2e437690ee7003c0f67360f0ab8d09d7ccf5360bd2fb3a7d6b9ee33c0894301b
Jul 15 14:29:48.143284 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:48.143192    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw" event={"ID":"cc26eca5-9e87-4ec1-b871-094342f5f1da","Type":"ContainerStarted","Data":"36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e"}
Jul 15 14:29:48.143284 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:48.143245    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw" event={"ID":"cc26eca5-9e87-4ec1-b871-094342f5f1da","Type":"ContainerStarted","Data":"2e437690ee7003c0f67360f0ab8d09d7ccf5360bd2fb3a7d6b9ee33c0894301b"}
Jul 15 14:29:48.143284 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:48.143275    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:48.189123 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:48.182753    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw" podStartSLOduration=1.8187170799999999 podStartE2EDuration="2.182729188s" totalImagesPullingTime="364.012108ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:29:48.176072258 +0000 UTC m=+2309.295877840" watchObservedRunningTime="2026-07-15 14:29:48.182729188 +0000 UTC m=+2309.302534773"
Jul 15 14:29:54.155221 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:54.155190    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:29:58.735033 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:58.734994    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"]
Jul 15 14:29:58.738500 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:58.738477    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"
Jul 15 14:29:58.746720 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:58.746690    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\""
Jul 15 14:29:58.747570 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:58.747539    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"]
Jul 15 14:29:58.833378 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:58.833327    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f702fcb4-4043-429a-ba6d-b16b1a3f070f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-752a1c-htbfv\" (UID: \"f702fcb4-4043-429a-ba6d-b16b1a3f070f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"
Jul 15 14:29:58.934792 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:58.934744    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f702fcb4-4043-429a-ba6d-b16b1a3f070f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-752a1c-htbfv\" (UID: \"f702fcb4-4043-429a-ba6d-b16b1a3f070f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"
Jul 15 14:29:58.935433 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:58.935413    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f702fcb4-4043-429a-ba6d-b16b1a3f070f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-752a1c-htbfv\" (UID: \"f702fcb4-4043-429a-ba6d-b16b1a3f070f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"
Jul 15 14:29:59.051416 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:59.051368    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"
Jul 15 14:29:59.204820 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:29:59.204796    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"]
Jul 15 14:29:59.206943 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:29:59.206906    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf702fcb4_4043_429a_ba6d_b16b1a3f070f.slice/crio-0773e22ed2c2062cf2005c54690a6de62e859789f95ed746f257dbc7cb023431 WatchSource:0}: Error finding container 0773e22ed2c2062cf2005c54690a6de62e859789f95ed746f257dbc7cb023431: Status 404 returned error can't find the container with id 0773e22ed2c2062cf2005c54690a6de62e859789f95ed746f257dbc7cb023431
Jul 15 14:30:00.135101 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.135064    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"]
Jul 15 14:30:00.138679 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.138644    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"
Jul 15 14:30:00.141451 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.141424    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45"]
Jul 15 14:30:00.144879 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.144858    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45"
Jul 15 14:30:00.149753 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.149685    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"
Jul 15 14:30:00.150573 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.150544    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"]
Jul 15 14:30:00.153839 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.153770    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45"]
Jul 15 14:30:00.162727 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.162696    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45"
Jul 15 14:30:00.209105 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.209071    2573 generic.go:356] "Generic (PLEG): container finished" podID="f702fcb4-4043-429a-ba6d-b16b1a3f070f" containerID="e9d06e0ad19d42a8275f106a992e627bd6f03183cca748445ae7815785acd3d3" exitCode=0
Jul 15 14:30:00.209274 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.209150    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv" event={"ID":"f702fcb4-4043-429a-ba6d-b16b1a3f070f","Type":"ContainerDied","Data":"e9d06e0ad19d42a8275f106a992e627bd6f03183cca748445ae7815785acd3d3"}
Jul 15 14:30:00.209274 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.209189    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv" event={"ID":"f702fcb4-4043-429a-ba6d-b16b1a3f070f","Type":"ContainerStarted","Data":"0773e22ed2c2062cf2005c54690a6de62e859789f95ed746f257dbc7cb023431"}
Jul 15 14:30:00.323636 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.323594    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45"]
Jul 15 14:30:00.325662 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:30:00.325635    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25139253_7492_4f7f_9992_37832c969880.slice/crio-020f3cf7e37bb317bd05276da81305588928a7d93f09de383d86c659bb128454 WatchSource:0}: Error finding container 020f3cf7e37bb317bd05276da81305588928a7d93f09de383d86c659bb128454: Status 404 returned error can't find the container with id 020f3cf7e37bb317bd05276da81305588928a7d93f09de383d86c659bb128454
Jul 15 14:30:00.346424 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:00.346390    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"]
Jul 15 14:30:00.347805 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:30:00.347778    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b140c64_0fad_4bdc_91b0_9fb3286554b1.slice/crio-0ac77bb74ad70f2f8a565b74ffe51264d86c34001de4dd5ab5e0dc49f1472d90 WatchSource:0}: Error finding container 0ac77bb74ad70f2f8a565b74ffe51264d86c34001de4dd5ab5e0dc49f1472d90: Status 404 returned error can't find the container with id 0ac77bb74ad70f2f8a565b74ffe51264d86c34001de4dd5ab5e0dc49f1472d90
Jul 15 14:30:01.217540 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.217507    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45" event={"ID":"25139253-7492-4f7f-9992-37832c969880","Type":"ContainerStarted","Data":"cb3be58a478e63b3f8509f8dc8cd8315d3659d16e2a47c208dadeb0d71bf3b3d"}
Jul 15 14:30:01.217934 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.217574    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45" event={"ID":"25139253-7492-4f7f-9992-37832c969880","Type":"ContainerStarted","Data":"020f3cf7e37bb317bd05276da81305588928a7d93f09de383d86c659bb128454"}
Jul 15 14:30:01.219359 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.219309    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd" event={"ID":"7b140c64-0fad-4bdc-91b0-9fb3286554b1","Type":"ContainerStarted","Data":"ff04607e6ee953ed364941a9e3b4054fb94bbe98159fee16f3b4fa67acbc6901"}
Jul 15 14:30:01.219482 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.219464    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd" event={"ID":"7b140c64-0fad-4bdc-91b0-9fb3286554b1","Type":"ContainerStarted","Data":"0ac77bb74ad70f2f8a565b74ffe51264d86c34001de4dd5ab5e0dc49f1472d90"}
Jul 15 14:30:01.236541 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.236482    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45" podStartSLOduration=1.2364662 podStartE2EDuration="1.2364662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:30:01.234587497 +0000 UTC m=+2322.354393080" watchObservedRunningTime="2026-07-15 14:30:01.2364662 +0000 UTC m=+2322.356271782"
Jul 15 14:30:01.374830 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.374801    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"
Jul 15 14:30:01.458974 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.458881    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f702fcb4-4043-429a-ba6d-b16b1a3f070f-maas-api-service-ca\") pod \"f702fcb4-4043-429a-ba6d-b16b1a3f070f\" (UID: \"f702fcb4-4043-429a-ba6d-b16b1a3f070f\") "
Jul 15 14:30:01.459238 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.459214    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f702fcb4-4043-429a-ba6d-b16b1a3f070f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f702fcb4-4043-429a-ba6d-b16b1a3f070f" (UID: "f702fcb4-4043-429a-ba6d-b16b1a3f070f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:30:01.559853 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:01.559771    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f702fcb4-4043-429a-ba6d-b16b1a3f070f-maas-api-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:30:02.227456 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:02.227422    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv" event={"ID":"f702fcb4-4043-429a-ba6d-b16b1a3f070f","Type":"ContainerDied","Data":"0773e22ed2c2062cf2005c54690a6de62e859789f95ed746f257dbc7cb023431"}
Jul 15 14:30:02.227882 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:02.227481    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0773e22ed2c2062cf2005c54690a6de62e859789f95ed746f257dbc7cb023431"
Jul 15 14:30:02.227882 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:02.227429    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-752a1c-htbfv"
Jul 15 14:30:02.229100 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:02.229067    2573 generic.go:356] "Generic (PLEG): container finished" podID="25139253-7492-4f7f-9992-37832c969880" containerID="cb3be58a478e63b3f8509f8dc8cd8315d3659d16e2a47c208dadeb0d71bf3b3d" exitCode=0
Jul 15 14:30:02.229234 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:02.229157    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45" event={"ID":"25139253-7492-4f7f-9992-37832c969880","Type":"ContainerDied","Data":"cb3be58a478e63b3f8509f8dc8cd8315d3659d16e2a47c208dadeb0d71bf3b3d"}
Jul 15 14:30:02.230944 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:02.230917    2573 generic.go:356] "Generic (PLEG): container finished" podID="7b140c64-0fad-4bdc-91b0-9fb3286554b1" containerID="ff04607e6ee953ed364941a9e3b4054fb94bbe98159fee16f3b4fa67acbc6901" exitCode=0
Jul 15 14:30:02.231055 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:02.230950    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd" event={"ID":"7b140c64-0fad-4bdc-91b0-9fb3286554b1","Type":"ContainerDied","Data":"ff04607e6ee953ed364941a9e3b4054fb94bbe98159fee16f3b4fa67acbc6901"}
Jul 15 14:30:02.367913 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:02.367882    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"
Jul 15 14:30:03.238123 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:03.238094    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"
Jul 15 14:30:03.238512 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:03.238126    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd" event={"ID":"7b140c64-0fad-4bdc-91b0-9fb3286554b1","Type":"ContainerDied","Data":"0ac77bb74ad70f2f8a565b74ffe51264d86c34001de4dd5ab5e0dc49f1472d90"}
Jul 15 14:30:03.238512 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:03.238181    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ac77bb74ad70f2f8a565b74ffe51264d86c34001de4dd5ab5e0dc49f1472d90"
Jul 15 14:30:03.384858 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:03.384827    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45"
Jul 15 14:30:04.244797 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:04.244768    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45"
Jul 15 14:30:04.245208 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:04.244799    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735430-d5n45" event={"ID":"25139253-7492-4f7f-9992-37832c969880","Type":"ContainerDied","Data":"020f3cf7e37bb317bd05276da81305588928a7d93f09de383d86c659bb128454"}
Jul 15 14:30:04.245208 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:04.244843    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="020f3cf7e37bb317bd05276da81305588928a7d93f09de383d86c659bb128454"
Jul 15 14:30:10.432665 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.432617    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"]
Jul 15 14:30:10.433264 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.432886    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg" podUID="edbf31ec-ad92-4301-b517-b13fd01215ce" containerName="maas-api" containerID="cri-o://445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b" gracePeriod=30
Jul 15 14:30:10.472479 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.472441    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"]
Jul 15 14:30:10.475922 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.475885    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-752a1c-29735430-z2fbd"]
Jul 15 14:30:10.684850 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.684761    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:30:10.848494 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.848449    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2l6f\" (UniqueName: \"kubernetes.io/projected/edbf31ec-ad92-4301-b517-b13fd01215ce-kube-api-access-m2l6f\") pod \"edbf31ec-ad92-4301-b517-b13fd01215ce\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") "
Jul 15 14:30:10.848697 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.848545    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbf31ec-ad92-4301-b517-b13fd01215ce-openshift-service-ca\") pod \"edbf31ec-ad92-4301-b517-b13fd01215ce\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") "
Jul 15 14:30:10.848697 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.848618    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls\") pod \"edbf31ec-ad92-4301-b517-b13fd01215ce\" (UID: \"edbf31ec-ad92-4301-b517-b13fd01215ce\") "
Jul 15 14:30:10.848958 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.848928    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edbf31ec-ad92-4301-b517-b13fd01215ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "edbf31ec-ad92-4301-b517-b13fd01215ce" (UID: "edbf31ec-ad92-4301-b517-b13fd01215ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:30:10.850713 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.850682    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "edbf31ec-ad92-4301-b517-b13fd01215ce" (UID: "edbf31ec-ad92-4301-b517-b13fd01215ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:30:10.850803 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.850736    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edbf31ec-ad92-4301-b517-b13fd01215ce-kube-api-access-m2l6f" (OuterVolumeSpecName: "kube-api-access-m2l6f") pod "edbf31ec-ad92-4301-b517-b13fd01215ce" (UID: "edbf31ec-ad92-4301-b517-b13fd01215ce"). InnerVolumeSpecName "kube-api-access-m2l6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:30:10.949953 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.949849    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2l6f\" (UniqueName: \"kubernetes.io/projected/edbf31ec-ad92-4301-b517-b13fd01215ce-kube-api-access-m2l6f\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:30:10.949953 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.949885    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbf31ec-ad92-4301-b517-b13fd01215ce-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:30:10.949953 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:10.949896    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbf31ec-ad92-4301-b517-b13fd01215ce-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:30:11.287680 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.287637    2573 generic.go:356] "Generic (PLEG): container finished" podID="edbf31ec-ad92-4301-b517-b13fd01215ce" containerID="445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b" exitCode=0
Jul 15 14:30:11.287864 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.287691    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"
Jul 15 14:30:11.287864 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.287701    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg" event={"ID":"edbf31ec-ad92-4301-b517-b13fd01215ce","Type":"ContainerDied","Data":"445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b"}
Jul 15 14:30:11.287864 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.287776    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg" event={"ID":"edbf31ec-ad92-4301-b517-b13fd01215ce","Type":"ContainerDied","Data":"0381168f650c7079e2fa16560dc54599b080674f77fe4f0572a7644aa8f7acde"}
Jul 15 14:30:11.287864 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.287811    2573 scope.go:117] "RemoveContainer" containerID="445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b"
Jul 15 14:30:11.298014 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.297991    2573 scope.go:117] "RemoveContainer" containerID="445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b"
Jul 15 14:30:11.298365 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:30:11.298327    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b\": container with ID starting with 445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b not found: ID does not exist" containerID="445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b"
Jul 15 14:30:11.298430 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.298379    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b"} err="failed to get container status \"445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b\": rpc error: code = NotFound desc = could not find container \"445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b\": container with ID starting with 445b2c6e8915ffb01508463c08adb91c803a5a7410ca527bdbd1eba605d7632b not found: ID does not exist"
Jul 15 14:30:11.315309 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.315273    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"]
Jul 15 14:30:11.320359 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.320304    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-752a1c-84f6876fcd-kwrwg"]
Jul 15 14:30:11.407033 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.406996    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b140c64-0fad-4bdc-91b0-9fb3286554b1" path="/var/lib/kubelet/pods/7b140c64-0fad-4bdc-91b0-9fb3286554b1/volumes"
Jul 15 14:30:11.407416 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:11.407401    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edbf31ec-ad92-4301-b517-b13fd01215ce" path="/var/lib/kubelet/pods/edbf31ec-ad92-4301-b517-b13fd01215ce/volumes"
Jul 15 14:30:35.588301 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.588261    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"]
Jul 15 14:30:35.588814 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.588587    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw" podUID="cc26eca5-9e87-4ec1-b871-094342f5f1da" containerName="maas-api" containerID="cri-o://36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e" gracePeriod=30
Jul 15 14:30:35.847379 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.847293    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:30:35.995465 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.995430    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls\") pod \"cc26eca5-9e87-4ec1-b871-094342f5f1da\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") "
Jul 15 14:30:35.995465 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.995466    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwjfc\" (UniqueName: \"kubernetes.io/projected/cc26eca5-9e87-4ec1-b871-094342f5f1da-kube-api-access-fwjfc\") pod \"cc26eca5-9e87-4ec1-b871-094342f5f1da\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") "
Jul 15 14:30:35.995698 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.995542    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc26eca5-9e87-4ec1-b871-094342f5f1da-openshift-service-ca\") pod \"cc26eca5-9e87-4ec1-b871-094342f5f1da\" (UID: \"cc26eca5-9e87-4ec1-b871-094342f5f1da\") "
Jul 15 14:30:35.995952 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.995926    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc26eca5-9e87-4ec1-b871-094342f5f1da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc26eca5-9e87-4ec1-b871-094342f5f1da" (UID: "cc26eca5-9e87-4ec1-b871-094342f5f1da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:30:35.997723 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.997699    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc26eca5-9e87-4ec1-b871-094342f5f1da" (UID: "cc26eca5-9e87-4ec1-b871-094342f5f1da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:30:35.997723 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:35.997712    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc26eca5-9e87-4ec1-b871-094342f5f1da-kube-api-access-fwjfc" (OuterVolumeSpecName: "kube-api-access-fwjfc") pod "cc26eca5-9e87-4ec1-b871-094342f5f1da" (UID: "cc26eca5-9e87-4ec1-b871-094342f5f1da"). InnerVolumeSpecName "kube-api-access-fwjfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:30:36.096517 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.096483    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwjfc\" (UniqueName: \"kubernetes.io/projected/cc26eca5-9e87-4ec1-b871-094342f5f1da-kube-api-access-fwjfc\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:30:36.096517 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.096514    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc26eca5-9e87-4ec1-b871-094342f5f1da-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:30:36.096517 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.096527    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc26eca5-9e87-4ec1-b871-094342f5f1da-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:30:36.423817 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.423776    2573 generic.go:356] "Generic (PLEG): container finished" podID="cc26eca5-9e87-4ec1-b871-094342f5f1da" containerID="36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e" exitCode=0
Jul 15 14:30:36.423817 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.423822    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"
Jul 15 14:30:36.424036 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.423827    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw" event={"ID":"cc26eca5-9e87-4ec1-b871-094342f5f1da","Type":"ContainerDied","Data":"36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e"}
Jul 15 14:30:36.424036 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.423881    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw" event={"ID":"cc26eca5-9e87-4ec1-b871-094342f5f1da","Type":"ContainerDied","Data":"2e437690ee7003c0f67360f0ab8d09d7ccf5360bd2fb3a7d6b9ee33c0894301b"}
Jul 15 14:30:36.424036 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.423910    2573 scope.go:117] "RemoveContainer" containerID="36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e"
Jul 15 14:30:36.434223 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.434202    2573 scope.go:117] "RemoveContainer" containerID="36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e"
Jul 15 14:30:36.434558 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:30:36.434536    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e\": container with ID starting with 36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e not found: ID does not exist" containerID="36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e"
Jul 15 14:30:36.434653 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.434572    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e"} err="failed to get container status \"36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e\": rpc error: code = NotFound desc = could not find container \"36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e\": container with ID starting with 36a776737aed69ffe5c80c60b58fcbe0bccbb5dcc75d833f13f2e175947e7b2e not found: ID does not exist"
Jul 15 14:30:36.450803 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.450762    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"]
Jul 15 14:30:36.452875 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:36.452850    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e7e87a-757cf775b5-2clqw"]
Jul 15 14:30:37.407116 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:37.407080    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc26eca5-9e87-4ec1-b871-094342f5f1da" path="/var/lib/kubelet/pods/cc26eca5-9e87-4ec1-b871-094342f5f1da/volumes"
Jul 15 14:30:54.453580 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453544    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"]
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453931    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc26eca5-9e87-4ec1-b871-094342f5f1da" containerName="maas-api"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453942    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc26eca5-9e87-4ec1-b871-094342f5f1da" containerName="maas-api"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453956    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f702fcb4-4043-429a-ba6d-b16b1a3f070f" containerName="revoke-keys"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453962    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f702fcb4-4043-429a-ba6d-b16b1a3f070f" containerName="revoke-keys"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453972    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edbf31ec-ad92-4301-b517-b13fd01215ce" containerName="maas-api"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453978    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="edbf31ec-ad92-4301-b517-b13fd01215ce" containerName="maas-api"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453984    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b140c64-0fad-4bdc-91b0-9fb3286554b1" containerName="cleanup"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.453990    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b140c64-0fad-4bdc-91b0-9fb3286554b1" containerName="cleanup"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.454007    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25139253-7492-4f7f-9992-37832c969880" containerName="cleanup"
Jul 15 14:30:54.454040 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.454012    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="25139253-7492-4f7f-9992-37832c969880" containerName="cleanup"
Jul 15 14:30:54.454391 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.454083    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc26eca5-9e87-4ec1-b871-094342f5f1da" containerName="maas-api"
Jul 15 14:30:54.454391 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.454091    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f702fcb4-4043-429a-ba6d-b16b1a3f070f" containerName="revoke-keys"
Jul 15 14:30:54.454391 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.454100    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="25139253-7492-4f7f-9992-37832c969880" containerName="cleanup"
Jul 15 14:30:54.454391 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.454106    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="edbf31ec-ad92-4301-b517-b13fd01215ce" containerName="maas-api"
Jul 15 14:30:54.454391 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.454112    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b140c64-0fad-4bdc-91b0-9fb3286554b1" containerName="cleanup"
Jul 15 14:30:54.458661 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.458631    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:54.464202 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.464176    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 14:30:54.464433 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.464244    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 14:30:54.464574 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.464453    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0b4a59-serving-cert\""
Jul 15 14:30:54.465268 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.465248    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\""
Jul 15 14:30:54.475115 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.475076    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f484c474-e9fd-4788-9ccc-253cf8c3af1b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:54.475295 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.475146    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99j7k\" (UniqueName: \"kubernetes.io/projected/f484c474-e9fd-4788-9ccc-253cf8c3af1b-kube-api-access-99j7k\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:54.475295 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.475261    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:54.477776 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.477752    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"]
Jul 15 14:30:54.575889 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.575848    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:54.576078 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.575923    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f484c474-e9fd-4788-9ccc-253cf8c3af1b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:54.576078 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.575951    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99j7k\" (UniqueName: \"kubernetes.io/projected/f484c474-e9fd-4788-9ccc-253cf8c3af1b-kube-api-access-99j7k\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:54.576078 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:30:54.576001    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-serving-cert: secret "maas-api-e2e-shared-a-0b4a59-serving-cert" not found
Jul 15 14:30:54.576198 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:30:54.576088    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls podName:f484c474-e9fd-4788-9ccc-253cf8c3af1b nodeName:}" failed. No retries permitted until 2026-07-15 14:30:55.076067122 +0000 UTC m=+2376.195872681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls") pod "maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8" (UID: "f484c474-e9fd-4788-9ccc-253cf8c3af1b") : secret "maas-api-e2e-shared-a-0b4a59-serving-cert" not found
Jul 15 14:30:54.576657 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.576635    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f484c474-e9fd-4788-9ccc-253cf8c3af1b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:54.586285 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:54.586257    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99j7k\" (UniqueName: \"kubernetes.io/projected/f484c474-e9fd-4788-9ccc-253cf8c3af1b-kube-api-access-99j7k\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:55.080185 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:55.080137    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:55.082638 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:55.082617    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls\") pod \"maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:55.369316 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:55.369213    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:55.515371 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:55.515328    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"]
Jul 15 14:30:55.517780 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:30:55.517752    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf484c474_e9fd_4788_9ccc_253cf8c3af1b.slice/crio-81cfc92107514e5b4964e711ff5ae5fabdb9579ea6564743b568225345afa1b6 WatchSource:0}: Error finding container 81cfc92107514e5b4964e711ff5ae5fabdb9579ea6564743b568225345afa1b6: Status 404 returned error can't find the container with id 81cfc92107514e5b4964e711ff5ae5fabdb9579ea6564743b568225345afa1b6
Jul 15 14:30:55.530171 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:55.530132    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8" event={"ID":"f484c474-e9fd-4788-9ccc-253cf8c3af1b","Type":"ContainerStarted","Data":"81cfc92107514e5b4964e711ff5ae5fabdb9579ea6564743b568225345afa1b6"}
Jul 15 14:30:56.537570 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:56.537528    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8" event={"ID":"f484c474-e9fd-4788-9ccc-253cf8c3af1b","Type":"ContainerStarted","Data":"962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66"}
Jul 15 14:30:56.538020 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:56.537706    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:30:56.557880 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:30:56.557817    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8" podStartSLOduration=1.8225149520000001 podStartE2EDuration="2.557797535s" totalImagesPullingTime="735.282583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:30:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:30:56.554856602 +0000 UTC m=+2377.674662184" watchObservedRunningTime="2026-07-15 14:30:56.557797535 +0000 UTC m=+2377.677603116"
Jul 15 14:31:02.554285 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.554254    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:31:02.777270 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.777233    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"]
Jul 15 14:31:02.781096 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.781072    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:02.783904 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.783882    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-0a4dfd-serving-cert\""
Jul 15 14:31:02.796050 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.796021    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"]
Jul 15 14:31:02.850270 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.850171    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z96k\" (UniqueName: \"kubernetes.io/projected/e19b273f-cbbf-4170-9a08-31c1d478e70f-kube-api-access-7z96k\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:02.850270 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.850225    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e19b273f-cbbf-4170-9a08-31c1d478e70f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:02.850270 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.850254    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:02.951034 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.950983    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z96k\" (UniqueName: \"kubernetes.io/projected/e19b273f-cbbf-4170-9a08-31c1d478e70f-kube-api-access-7z96k\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:02.951244 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.951047    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e19b273f-cbbf-4170-9a08-31c1d478e70f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:02.951244 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.951092    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:02.951244 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:31:02.951210    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-serving-cert: secret "maas-api-e2e-shared-b-0a4dfd-serving-cert" not found
Jul 15 14:31:02.951495 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:31:02.951311    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls podName:e19b273f-cbbf-4170-9a08-31c1d478e70f nodeName:}" failed. No retries permitted until 2026-07-15 14:31:03.451288327 +0000 UTC m=+2384.571093889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls") pod "maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28" (UID: "e19b273f-cbbf-4170-9a08-31c1d478e70f") : secret "maas-api-e2e-shared-b-0a4dfd-serving-cert" not found
Jul 15 14:31:02.951736 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.951715    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e19b273f-cbbf-4170-9a08-31c1d478e70f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:02.970725 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:02.970694    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z96k\" (UniqueName: \"kubernetes.io/projected/e19b273f-cbbf-4170-9a08-31c1d478e70f-kube-api-access-7z96k\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:03.457025 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:03.456979    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:03.459489 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:03.459464    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls\") pod \"maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:03.691929 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:03.691893    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:03.843764 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:03.843737    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"]
Jul 15 14:31:03.846442 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:31:03.846405    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19b273f_cbbf_4170_9a08_31c1d478e70f.slice/crio-31377511b922c1a1a876d413debcc1c1f092fd276b96cedad3996d5da153fb1e WatchSource:0}: Error finding container 31377511b922c1a1a876d413debcc1c1f092fd276b96cedad3996d5da153fb1e: Status 404 returned error can't find the container with id 31377511b922c1a1a876d413debcc1c1f092fd276b96cedad3996d5da153fb1e
Jul 15 14:31:04.589001 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:04.588964    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28" event={"ID":"e19b273f-cbbf-4170-9a08-31c1d478e70f","Type":"ContainerStarted","Data":"d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00"}
Jul 15 14:31:04.589202 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:04.589021    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28" event={"ID":"e19b273f-cbbf-4170-9a08-31c1d478e70f","Type":"ContainerStarted","Data":"31377511b922c1a1a876d413debcc1c1f092fd276b96cedad3996d5da153fb1e"}
Jul 15 14:31:04.589202 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:04.589097    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:04.612323 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:04.612268    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28" podStartSLOduration=1.9859648 podStartE2EDuration="2.612247686s" totalImagesPullingTime="626.282886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:31:04.610456897 +0000 UTC m=+2385.730262479" watchObservedRunningTime="2026-07-15 14:31:04.612247686 +0000 UTC m=+2385.732053266"
Jul 15 14:31:10.599372 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:10.599314    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:31:19.554241 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:19.554203    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:31:19.556820 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:19.556797    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:31:23.695275 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.695235    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"]
Jul 15 14:31:23.700734 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.700699    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.703638 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.703608    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"openshift-service-ca.crt\""
Jul 15 14:31:23.704806 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.704781    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"auth-test-model-0b4a59-kserve-self-signed-certs\""
Jul 15 14:31:23.704926 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.704789    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"kube-root-ca.crt\""
Jul 15 14:31:23.704992 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.704974    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"default-dockercfg-d67p6\""
Jul 15 14:31:23.709402 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.709373    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"]
Jul 15 14:31:23.862319 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.862274    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-model-cache\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.862319 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.862318    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-home\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.862567 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.862360    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-tmp-dir\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.862567 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.862429    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plm7k\" (UniqueName: \"kubernetes.io/projected/0009ac2e-2012-4513-9b03-581db259156b-kube-api-access-plm7k\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.862567 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.862491    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0009ac2e-2012-4513-9b03-581db259156b-tls-certs\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.862567 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.862512    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-dshm\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.963466 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963355    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-model-cache\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.963466 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963411    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-home\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.963466 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963437    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-tmp-dir\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.963466 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963463    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plm7k\" (UniqueName: \"kubernetes.io/projected/0009ac2e-2012-4513-9b03-581db259156b-kube-api-access-plm7k\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.963763 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963512    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0009ac2e-2012-4513-9b03-581db259156b-tls-certs\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.963763 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963532    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-dshm\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.963886 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963787    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-model-cache\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.963886 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963844    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-home\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.964007 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.963915    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-tmp-dir\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.965919 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.965886    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-dshm\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.966085 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.966056    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0009ac2e-2012-4513-9b03-581db259156b-tls-certs\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:23.972964 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:23.972932    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plm7k\" (UniqueName: \"kubernetes.io/projected/0009ac2e-2012-4513-9b03-581db259156b-kube-api-access-plm7k\") pod \"auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") " pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:24.013103 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:24.013006    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:24.162867 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:24.162842    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"]
Jul 15 14:31:24.164723 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:31:24.164690    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0009ac2e_2012_4513_9b03_581db259156b.slice/crio-e55e630cafbcddc8db6dbe498a2945f40393ec8e7e8b2f8d4c30a87883e86cd9 WatchSource:0}: Error finding container e55e630cafbcddc8db6dbe498a2945f40393ec8e7e8b2f8d4c30a87883e86cd9: Status 404 returned error can't find the container with id e55e630cafbcddc8db6dbe498a2945f40393ec8e7e8b2f8d4c30a87883e86cd9
Jul 15 14:31:24.702662 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:24.702612    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv" event={"ID":"0009ac2e-2012-4513-9b03-581db259156b","Type":"ContainerStarted","Data":"afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a"}
Jul 15 14:31:24.702662 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:24.702665    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv" event={"ID":"0009ac2e-2012-4513-9b03-581db259156b","Type":"ContainerStarted","Data":"e55e630cafbcddc8db6dbe498a2945f40393ec8e7e8b2f8d4c30a87883e86cd9"}
Jul 15 14:31:24.722938 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:24.722881    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv" podStartSLOduration=1.722862943 podStartE2EDuration="1.722862943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:31:24.721851744 +0000 UTC m=+2405.841657359" watchObservedRunningTime="2026-07-15 14:31:24.722862943 +0000 UTC m=+2405.842668521"
Jul 15 14:31:34.014100 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:34.014048    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:34.014100 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:34.014114    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:34.026865 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:34.026838    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:31:34.769505 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:31:34.769467    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:34:31.524906 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:31.524868    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"]
Jul 15 14:34:46.545617 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.545574    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv" podUID="0009ac2e-2012-4513-9b03-581db259156b" containerName="main" containerID="cri-o://afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a" gracePeriod=45
Jul 15 14:34:46.802461 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.802389    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:34:46.804664 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.804635    2573 generic.go:356] "Generic (PLEG): container finished" podID="0009ac2e-2012-4513-9b03-581db259156b" containerID="afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a" exitCode=0
Jul 15 14:34:46.804789 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.804667    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"
Jul 15 14:34:46.804789 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.804678    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv" event={"ID":"0009ac2e-2012-4513-9b03-581db259156b","Type":"ContainerDied","Data":"afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a"}
Jul 15 14:34:46.804789 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.804708    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv" event={"ID":"0009ac2e-2012-4513-9b03-581db259156b","Type":"ContainerDied","Data":"e55e630cafbcddc8db6dbe498a2945f40393ec8e7e8b2f8d4c30a87883e86cd9"}
Jul 15 14:34:46.804789 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.804724    2573 scope.go:117] "RemoveContainer" containerID="afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a"
Jul 15 14:34:46.814414 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.814394    2573 scope.go:117] "RemoveContainer" containerID="afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a"
Jul 15 14:34:46.814752 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:34:46.814730    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a\": container with ID starting with afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a not found: ID does not exist" containerID="afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a"
Jul 15 14:34:46.814816 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.814762    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a"} err="failed to get container status \"afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a\": rpc error: code = NotFound desc = could not find container \"afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a\": container with ID starting with afcade23501b2c9fab38473b1bc193961c02450e95e973ea726ad9bb18fe811a not found: ID does not exist"
Jul 15 14:34:46.853362 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853303    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-dshm\") pod \"0009ac2e-2012-4513-9b03-581db259156b\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") "
Jul 15 14:34:46.853362 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853371    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-model-cache\") pod \"0009ac2e-2012-4513-9b03-581db259156b\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") "
Jul 15 14:34:46.853618 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853397    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-home\") pod \"0009ac2e-2012-4513-9b03-581db259156b\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") "
Jul 15 14:34:46.853618 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853430    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plm7k\" (UniqueName: \"kubernetes.io/projected/0009ac2e-2012-4513-9b03-581db259156b-kube-api-access-plm7k\") pod \"0009ac2e-2012-4513-9b03-581db259156b\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") "
Jul 15 14:34:46.853618 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853478    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0009ac2e-2012-4513-9b03-581db259156b-tls-certs\") pod \"0009ac2e-2012-4513-9b03-581db259156b\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") "
Jul 15 14:34:46.853618 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853545    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-tmp-dir\") pod \"0009ac2e-2012-4513-9b03-581db259156b\" (UID: \"0009ac2e-2012-4513-9b03-581db259156b\") "
Jul 15 14:34:46.853834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853656    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-model-cache" (OuterVolumeSpecName: "model-cache") pod "0009ac2e-2012-4513-9b03-581db259156b" (UID: "0009ac2e-2012-4513-9b03-581db259156b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:34:46.853834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853693    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-home" (OuterVolumeSpecName: "home") pod "0009ac2e-2012-4513-9b03-581db259156b" (UID: "0009ac2e-2012-4513-9b03-581db259156b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:34:46.853834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853820    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0009ac2e-2012-4513-9b03-581db259156b" (UID: "0009ac2e-2012-4513-9b03-581db259156b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:34:46.853987 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853930    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-tmp-dir\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:34:46.853987 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853947    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-model-cache\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:34:46.853987 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.853956    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-home\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:34:46.855734 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.855699    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0009ac2e-2012-4513-9b03-581db259156b-kube-api-access-plm7k" (OuterVolumeSpecName: "kube-api-access-plm7k") pod "0009ac2e-2012-4513-9b03-581db259156b" (UID: "0009ac2e-2012-4513-9b03-581db259156b"). InnerVolumeSpecName "kube-api-access-plm7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:34:46.855734 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.855707    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0009ac2e-2012-4513-9b03-581db259156b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0009ac2e-2012-4513-9b03-581db259156b" (UID: "0009ac2e-2012-4513-9b03-581db259156b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:34:46.855903 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.855752    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-dshm" (OuterVolumeSpecName: "dshm") pod "0009ac2e-2012-4513-9b03-581db259156b" (UID: "0009ac2e-2012-4513-9b03-581db259156b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:34:46.954570 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.954526    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plm7k\" (UniqueName: \"kubernetes.io/projected/0009ac2e-2012-4513-9b03-581db259156b-kube-api-access-plm7k\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:34:46.954570 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.954562    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0009ac2e-2012-4513-9b03-581db259156b-tls-certs\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:34:46.954570 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:46.954573    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0009ac2e-2012-4513-9b03-581db259156b-dshm\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:34:47.159775 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:47.159736    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"]
Jul 15 14:34:47.165416 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:47.165379    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/auth-test-model-0b4a59-kserve-965bcd4c5-g6bdv"]
Jul 15 14:34:47.409606 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:34:47.409505    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0009ac2e-2012-4513-9b03-581db259156b" path="/var/lib/kubelet/pods/0009ac2e-2012-4513-9b03-581db259156b/volumes"
Jul 15 14:35:03.921776 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.921735    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"]
Jul 15 14:35:03.922382 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.922152    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0009ac2e-2012-4513-9b03-581db259156b" containerName="main"
Jul 15 14:35:03.922382 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.922164    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0009ac2e-2012-4513-9b03-581db259156b" containerName="main"
Jul 15 14:35:03.922382 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.922244    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0009ac2e-2012-4513-9b03-581db259156b" containerName="main"
Jul 15 14:35:03.925692 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.925666    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:03.929323 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.929298    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"default-dockercfg-d67p6\""
Jul 15 14:35:03.930503 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.930487    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"sub-test-model-0b4a59-kserve-self-signed-certs\""
Jul 15 14:35:03.930616 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.930562    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"kube-root-ca.crt\""
Jul 15 14:35:03.930616 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.930588    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"openshift-service-ca.crt\""
Jul 15 14:35:03.937468 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:03.937441    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"]
Jul 15 14:35:04.023487 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.023446    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-model-cache\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.023712 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.023494    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v567w\" (UniqueName: \"kubernetes.io/projected/70f90ac5-27e7-46e9-98bd-ded0512d475b-kube-api-access-v567w\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.023712 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.023589    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-home\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.023712 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.023667    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f90ac5-27e7-46e9-98bd-ded0512d475b-tls-certs\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.023712 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.023702    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-tmp-dir\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.023867 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.023730    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-dshm\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.124747 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.124705    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-model-cache\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.124747 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.124750    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v567w\" (UniqueName: \"kubernetes.io/projected/70f90ac5-27e7-46e9-98bd-ded0512d475b-kube-api-access-v567w\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.125024 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.124781    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-home\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.125024 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.124819    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f90ac5-27e7-46e9-98bd-ded0512d475b-tls-certs\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.125024 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.124844    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-tmp-dir\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.125024 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.124863    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-dshm\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.125214 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.125189    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-model-cache\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.125263 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.125219    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-tmp-dir\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.125263 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.125249    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-home\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.127239 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.127206    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-dshm\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.127448 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.127429    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f90ac5-27e7-46e9-98bd-ded0512d475b-tls-certs\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.137212 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.137181    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v567w\" (UniqueName: \"kubernetes.io/projected/70f90ac5-27e7-46e9-98bd-ded0512d475b-kube-api-access-v567w\") pod \"sub-test-model-0b4a59-kserve-55dbc5985-9cjd2\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") " pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.235976 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.235873    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:04.382903 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.382833    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"]
Jul 15 14:35:04.385299 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:35:04.385271    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70f90ac5_27e7_46e9_98bd_ded0512d475b.slice/crio-169b6d86aa8d444b7f969040eba253b5327f29518aaf7b07b54f8f6bc3249f5f WatchSource:0}: Error finding container 169b6d86aa8d444b7f969040eba253b5327f29518aaf7b07b54f8f6bc3249f5f: Status 404 returned error can't find the container with id 169b6d86aa8d444b7f969040eba253b5327f29518aaf7b07b54f8f6bc3249f5f
Jul 15 14:35:04.902766 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.902719    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2" event={"ID":"70f90ac5-27e7-46e9-98bd-ded0512d475b","Type":"ContainerStarted","Data":"f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091"}
Jul 15 14:35:04.902958 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.902775    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2" event={"ID":"70f90ac5-27e7-46e9-98bd-ded0512d475b","Type":"ContainerStarted","Data":"169b6d86aa8d444b7f969040eba253b5327f29518aaf7b07b54f8f6bc3249f5f"}
Jul 15 14:35:04.922782 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:04.922728    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2" podStartSLOduration=1.9227098040000001 podStartE2EDuration="1.922709804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:35:04.921929681 +0000 UTC m=+2626.041735264" watchObservedRunningTime="2026-07-15 14:35:04.922709804 +0000 UTC m=+2626.042515384"
Jul 15 14:35:14.236834 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:14.236788    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:14.237236 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:14.237071    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:14.249754 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:14.249717    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:35:14.972620 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:35:14.972579    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:36:12.324152 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:12.324070    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"]
Jul 15 14:36:19.588407 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:19.588378    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:36:19.593266 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:19.593237    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mwc8d_2520d340-bfc2-4022-ad9b-4956b7a7b028/ovn-acl-logging/0.log"
Jul 15 14:36:19.678866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:19.678835    2573 scope.go:117] "RemoveContainer" containerID="ff04607e6ee953ed364941a9e3b4054fb94bbe98159fee16f3b4fa67acbc6901"
Jul 15 14:36:27.344455 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.344388    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2" podUID="70f90ac5-27e7-46e9-98bd-ded0512d475b" containerName="main" containerID="cri-o://f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091" gracePeriod=45
Jul 15 14:36:27.605551 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.605482    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:36:27.740130 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740090    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-model-cache\") pod \"70f90ac5-27e7-46e9-98bd-ded0512d475b\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") "
Jul 15 14:36:27.740376 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740151    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-tmp-dir\") pod \"70f90ac5-27e7-46e9-98bd-ded0512d475b\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") "
Jul 15 14:36:27.740376 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740177    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-dshm\") pod \"70f90ac5-27e7-46e9-98bd-ded0512d475b\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") "
Jul 15 14:36:27.740376 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740203    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f90ac5-27e7-46e9-98bd-ded0512d475b-tls-certs\") pod \"70f90ac5-27e7-46e9-98bd-ded0512d475b\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") "
Jul 15 14:36:27.740376 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740261    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v567w\" (UniqueName: \"kubernetes.io/projected/70f90ac5-27e7-46e9-98bd-ded0512d475b-kube-api-access-v567w\") pod \"70f90ac5-27e7-46e9-98bd-ded0512d475b\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") "
Jul 15 14:36:27.740376 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740293    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-home\") pod \"70f90ac5-27e7-46e9-98bd-ded0512d475b\" (UID: \"70f90ac5-27e7-46e9-98bd-ded0512d475b\") "
Jul 15 14:36:27.740653 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740515    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-model-cache" (OuterVolumeSpecName: "model-cache") pod "70f90ac5-27e7-46e9-98bd-ded0512d475b" (UID: "70f90ac5-27e7-46e9-98bd-ded0512d475b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:36:27.740653 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740529    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "70f90ac5-27e7-46e9-98bd-ded0512d475b" (UID: "70f90ac5-27e7-46e9-98bd-ded0512d475b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:36:27.740722 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740651    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-model-cache\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:36:27.740722 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740702    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-tmp-dir\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:36:27.740786 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.740722    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-home" (OuterVolumeSpecName: "home") pod "70f90ac5-27e7-46e9-98bd-ded0512d475b" (UID: "70f90ac5-27e7-46e9-98bd-ded0512d475b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:36:27.742457 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.742424    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70f90ac5-27e7-46e9-98bd-ded0512d475b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "70f90ac5-27e7-46e9-98bd-ded0512d475b" (UID: "70f90ac5-27e7-46e9-98bd-ded0512d475b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:36:27.742582 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.742424    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-dshm" (OuterVolumeSpecName: "dshm") pod "70f90ac5-27e7-46e9-98bd-ded0512d475b" (UID: "70f90ac5-27e7-46e9-98bd-ded0512d475b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:36:27.742582 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.742567    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f90ac5-27e7-46e9-98bd-ded0512d475b-kube-api-access-v567w" (OuterVolumeSpecName: "kube-api-access-v567w") pod "70f90ac5-27e7-46e9-98bd-ded0512d475b" (UID: "70f90ac5-27e7-46e9-98bd-ded0512d475b"). InnerVolumeSpecName "kube-api-access-v567w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:36:27.842113 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.842070    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-dshm\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:36:27.842113 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.842104    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f90ac5-27e7-46e9-98bd-ded0512d475b-tls-certs\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:36:27.842113 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.842119    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v567w\" (UniqueName: \"kubernetes.io/projected/70f90ac5-27e7-46e9-98bd-ded0512d475b-kube-api-access-v567w\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:36:27.842388 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:27.842132    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f90ac5-27e7-46e9-98bd-ded0512d475b-home\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:36:28.367594 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.367555    2573 generic.go:356] "Generic (PLEG): container finished" podID="70f90ac5-27e7-46e9-98bd-ded0512d475b" containerID="f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091" exitCode=0
Jul 15 14:36:28.368020 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.367612    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"
Jul 15 14:36:28.368020 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.367639    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2" event={"ID":"70f90ac5-27e7-46e9-98bd-ded0512d475b","Type":"ContainerDied","Data":"f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091"}
Jul 15 14:36:28.368020 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.367676    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2" event={"ID":"70f90ac5-27e7-46e9-98bd-ded0512d475b","Type":"ContainerDied","Data":"169b6d86aa8d444b7f969040eba253b5327f29518aaf7b07b54f8f6bc3249f5f"}
Jul 15 14:36:28.368020 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.367693    2573 scope.go:117] "RemoveContainer" containerID="f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091"
Jul 15 14:36:28.376644 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.376623    2573 scope.go:117] "RemoveContainer" containerID="f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091"
Jul 15 14:36:28.376933 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:36:28.376916    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091\": container with ID starting with f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091 not found: ID does not exist" containerID="f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091"
Jul 15 14:36:28.376981 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.376943    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091"} err="failed to get container status \"f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091\": rpc error: code = NotFound desc = could not find container \"f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091\": container with ID starting with f686566c56b547652ba106aa98070c678ba9fe3d02c2dd63c9763513ef4de091 not found: ID does not exist"
Jul 15 14:36:28.390518 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.390494    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"]
Jul 15 14:36:28.392406 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:28.392388    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/sub-test-model-0b4a59-kserve-55dbc5985-9cjd2"]
Jul 15 14:36:29.407033 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:29.406993    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70f90ac5-27e7-46e9-98bd-ded0512d475b" path="/var/lib/kubelet/pods/70f90ac5-27e7-46e9-98bd-ded0512d475b/volumes"
Jul 15 14:36:40.349301 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.349262    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"]
Jul 15 14:36:40.349744 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.349729    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f90ac5-27e7-46e9-98bd-ded0512d475b" containerName="main"
Jul 15 14:36:40.349744 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.349745    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f90ac5-27e7-46e9-98bd-ded0512d475b" containerName="main"
Jul 15 14:36:40.349820 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.349808    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="70f90ac5-27e7-46e9-98bd-ded0512d475b" containerName="main"
Jul 15 14:36:40.354618 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.354592    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.358009 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.357984    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"openshift-service-ca.crt\""
Jul 15 14:36:40.358133 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.358024    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"kube-root-ca.crt\""
Jul 15 14:36:40.359170 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.359145    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"rate-test-model-0b4a59-kserve-self-signed-certs\""
Jul 15 14:36:40.359403 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.359387    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0b4a59\"/\"default-dockercfg-d67p6\""
Jul 15 14:36:40.373470 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.373431    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"]
Jul 15 14:36:40.465236 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.465196    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tls-certs\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.465236 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.465240    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-model-cache\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.465503 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.465274    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-dshm\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.465503 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.465399    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb776\" (UniqueName: \"kubernetes.io/projected/79718661-d5a4-4b7f-89ad-7908bb7f0b37-kube-api-access-xb776\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.465503 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.465496    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tmp-dir\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.465603 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.465534    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-home\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.566486 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.566447    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb776\" (UniqueName: \"kubernetes.io/projected/79718661-d5a4-4b7f-89ad-7908bb7f0b37-kube-api-access-xb776\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.566701 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.566537    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tmp-dir\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.566701 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.566566    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-home\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.566701 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.566591    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tls-certs\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.566701 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.566612    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-model-cache\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.566921 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.566756    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-dshm\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.566997 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.566981    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-model-cache\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.567062 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.567039    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tmp-dir\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.567107 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.567080    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-home\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.569029 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.569003    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-dshm\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.569172 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.569150    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tls-certs\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.578390 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.578363    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb776\" (UniqueName: \"kubernetes.io/projected/79718661-d5a4-4b7f-89ad-7908bb7f0b37-kube-api-access-xb776\") pod \"rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") " pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.664717 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.664624    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:40.819723 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:36:40.819684    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79718661_d5a4_4b7f_89ad_7908bb7f0b37.slice/crio-069460ca1d13adea9eb831573f9cf3291cc8bffc7ec534140164a6d2d5a54da4 WatchSource:0}: Error finding container 069460ca1d13adea9eb831573f9cf3291cc8bffc7ec534140164a6d2d5a54da4: Status 404 returned error can't find the container with id 069460ca1d13adea9eb831573f9cf3291cc8bffc7ec534140164a6d2d5a54da4
Jul 15 14:36:40.832479 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:40.832448    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"]
Jul 15 14:36:41.438506 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:41.438469    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr" event={"ID":"79718661-d5a4-4b7f-89ad-7908bb7f0b37","Type":"ContainerStarted","Data":"e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f"}
Jul 15 14:36:41.438506 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:41.438511    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr" event={"ID":"79718661-d5a4-4b7f-89ad-7908bb7f0b37","Type":"ContainerStarted","Data":"069460ca1d13adea9eb831573f9cf3291cc8bffc7ec534140164a6d2d5a54da4"}
Jul 15 14:36:41.458529 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:41.458470    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr" podStartSLOduration=1.458449916 podStartE2EDuration="1.458449916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:36:41.457054445 +0000 UTC m=+2722.576860029" watchObservedRunningTime="2026-07-15 14:36:41.458449916 +0000 UTC m=+2722.578255496"
Jul 15 14:36:50.665008 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:50.664968    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:50.665665 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:50.665060    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:50.677995 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:50.677965    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:36:51.504662 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:36:51.504625    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:37:43.999294 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:43.999260    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"]
Jul 15 14:37:51.909738 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:51.909703    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:51.912741 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:51.912725    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:51.915388 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:51.915369    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5rt4r\""
Jul 15 14:37:51.921261 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:51.921240    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:52.047059 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:52.047021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zt5m\" (UniqueName: \"kubernetes.io/projected/fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef-kube-api-access-7zt5m\") pod \"test-curl-7772-140622177815552\" (UID: \"fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef\") " pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:52.147611 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:52.147573    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zt5m\" (UniqueName: \"kubernetes.io/projected/fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef-kube-api-access-7zt5m\") pod \"test-curl-7772-140622177815552\" (UID: \"fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef\") " pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:52.156659 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:52.156636    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zt5m\" (UniqueName: \"kubernetes.io/projected/fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef-kube-api-access-7zt5m\") pod \"test-curl-7772-140622177815552\" (UID: \"fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef\") " pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:52.223952 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:52.223866    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:52.355089 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:52.355062    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:52.357543 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:37:52.357510    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbd0cb01_c9ea_4dd7_a5fe_9e474b325eef.slice/crio-b2ee8dd529ad3aaf9a7bd96f29646adbc65abfc7a9217e79f533e6b0b2e84696 WatchSource:0}: Error finding container b2ee8dd529ad3aaf9a7bd96f29646adbc65abfc7a9217e79f533e6b0b2e84696: Status 404 returned error can't find the container with id b2ee8dd529ad3aaf9a7bd96f29646adbc65abfc7a9217e79f533e6b0b2e84696
Jul 15 14:37:52.359166 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:52.359149    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 14:37:52.842987 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:52.842949    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177815552" event={"ID":"fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef","Type":"ContainerStarted","Data":"b2ee8dd529ad3aaf9a7bd96f29646adbc65abfc7a9217e79f533e6b0b2e84696"}
Jul 15 14:37:53.853617 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:53.853582    2573 generic.go:356] "Generic (PLEG): container finished" podID="fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef" containerID="7ed3aea76457148eb467152cac96ae672ff3a4aa076321d22cec4df986ff00a3" exitCode=0
Jul 15 14:37:53.854003 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:53.853622    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177815552" event={"ID":"fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef","Type":"ContainerDied","Data":"7ed3aea76457148eb467152cac96ae672ff3a4aa076321d22cec4df986ff00a3"}
Jul 15 14:37:55.000155 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.000126    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:55.047787 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.047753    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7772-140622177815552_fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef/test-curl-7772-140622177815552/0.log"
Jul 15 14:37:55.072402 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.072365    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:55.074418 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.074389    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zt5m\" (UniqueName: \"kubernetes.io/projected/fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef-kube-api-access-7zt5m\") pod \"fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef\" (UID: \"fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef\") "
Jul 15 14:37:55.076620 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.076582    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef-kube-api-access-7zt5m" (OuterVolumeSpecName: "kube-api-access-7zt5m") pod "fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef" (UID: "fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef"). InnerVolumeSpecName "kube-api-access-7zt5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:37:55.076724 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.076594    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:55.175783 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.175707    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zt5m\" (UniqueName: \"kubernetes.io/projected/fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef-kube-api-access-7zt5m\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:37:55.203364 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.203318    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:55.203812 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.203794    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef" containerName="test-curl-7772-140622177815552"
Jul 15 14:37:55.203904 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.203816    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef" containerName="test-curl-7772-140622177815552"
Jul 15 14:37:55.203965 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.203944    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef" containerName="test-curl-7772-140622177815552"
Jul 15 14:37:55.207038 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.207017    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:55.216088 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.216057    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:55.277267 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.277222    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7wfl\" (UniqueName: \"kubernetes.io/projected/670486e1-86b4-41e4-bfcd-224ed244865c-kube-api-access-z7wfl\") pod \"test-curl-7772-140622177815552\" (UID: \"670486e1-86b4-41e4-bfcd-224ed244865c\") " pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:55.378492 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.378459    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7wfl\" (UniqueName: \"kubernetes.io/projected/670486e1-86b4-41e4-bfcd-224ed244865c-kube-api-access-z7wfl\") pod \"test-curl-7772-140622177815552\" (UID: \"670486e1-86b4-41e4-bfcd-224ed244865c\") " pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:55.388296 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.388270    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7wfl\" (UniqueName: \"kubernetes.io/projected/670486e1-86b4-41e4-bfcd-224ed244865c-kube-api-access-z7wfl\") pod \"test-curl-7772-140622177815552\" (UID: \"670486e1-86b4-41e4-bfcd-224ed244865c\") " pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:55.406954 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.406923    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef" path="/var/lib/kubelet/pods/fbd0cb01-c9ea-4dd7-a5fe-9e474b325eef/volumes"
Jul 15 14:37:55.517667 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.517583    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:55.659792 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.659767    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:55.662155 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:37:55.662127    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod670486e1_86b4_41e4_bfcd_224ed244865c.slice/crio-433ce575a92a5736a35e2efb822b63331f9a6ee9b65f72f8764f2ace5b63dac4 WatchSource:0}: Error finding container 433ce575a92a5736a35e2efb822b63331f9a6ee9b65f72f8764f2ace5b63dac4: Status 404 returned error can't find the container with id 433ce575a92a5736a35e2efb822b63331f9a6ee9b65f72f8764f2ace5b63dac4
Jul 15 14:37:55.865178 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.865142    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177815552" event={"ID":"670486e1-86b4-41e4-bfcd-224ed244865c","Type":"ContainerStarted","Data":"433ce575a92a5736a35e2efb822b63331f9a6ee9b65f72f8764f2ace5b63dac4"}
Jul 15 14:37:55.866329 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.866310    2573 scope.go:117] "RemoveContainer" containerID="7ed3aea76457148eb467152cac96ae672ff3a4aa076321d22cec4df986ff00a3"
Jul 15 14:37:55.866495 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:55.866358    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:56.872647 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:56.872610    2573 generic.go:356] "Generic (PLEG): container finished" podID="670486e1-86b4-41e4-bfcd-224ed244865c" containerID="93337c7f3f0d050d32c7622b2c4cca4541ba98416dbe063223a8ce4539b16ac6" exitCode=0
Jul 15 14:37:56.873089 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:56.872704    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177815552" event={"ID":"670486e1-86b4-41e4-bfcd-224ed244865c","Type":"ContainerDied","Data":"93337c7f3f0d050d32c7622b2c4cca4541ba98416dbe063223a8ce4539b16ac6"}
Jul 15 14:37:58.019839 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.019813    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:58.044120 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.044086    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7772-140622177815552_670486e1-86b4-41e4-bfcd-224ed244865c/test-curl-7772-140622177815552/0.log"
Jul 15 14:37:58.076541 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.076495    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:58.076878 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.076843    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7772-140622177815552"]
Jul 15 14:37:58.106672 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.106632    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7wfl\" (UniqueName: \"kubernetes.io/projected/670486e1-86b4-41e4-bfcd-224ed244865c-kube-api-access-z7wfl\") pod \"670486e1-86b4-41e4-bfcd-224ed244865c\" (UID: \"670486e1-86b4-41e4-bfcd-224ed244865c\") "
Jul 15 14:37:58.108678 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.108654    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/670486e1-86b4-41e4-bfcd-224ed244865c-kube-api-access-z7wfl" (OuterVolumeSpecName: "kube-api-access-z7wfl") pod "670486e1-86b4-41e4-bfcd-224ed244865c" (UID: "670486e1-86b4-41e4-bfcd-224ed244865c"). InnerVolumeSpecName "kube-api-access-z7wfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:37:58.207597 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.207504    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7wfl\" (UniqueName: \"kubernetes.io/projected/670486e1-86b4-41e4-bfcd-224ed244865c-kube-api-access-z7wfl\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:37:58.886629 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.886591    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="433ce575a92a5736a35e2efb822b63331f9a6ee9b65f72f8764f2ace5b63dac4"
Jul 15 14:37:58.886629 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:58.886614    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:58.897193 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:37:58.897155    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7772-140622177815552\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" podUID="670486e1-86b4-41e4-bfcd-224ed244865c" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:59.019988 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.019935    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr" podUID="79718661-d5a4-4b7f-89ad-7908bb7f0b37" containerName="main" containerID="cri-o://e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f" gracePeriod=45
Jul 15 14:37:59.280604 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.280574    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:37:59.301455 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:37:59.301418    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7772-140622177815552\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" podUID="670486e1-86b4-41e4-bfcd-224ed244865c" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:59.405737 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:37:59.405647    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7772-140622177815552\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" podUID="670486e1-86b4-41e4-bfcd-224ed244865c" pod="opendatahub/test-curl-7772-140622177815552"
Jul 15 14:37:59.406576 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.406557    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="670486e1-86b4-41e4-bfcd-224ed244865c" path="/var/lib/kubelet/pods/670486e1-86b4-41e4-bfcd-224ed244865c/volumes"
Jul 15 14:37:59.419580 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.419557    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tls-certs\") pod \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") "
Jul 15 14:37:59.419649 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.419614    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-model-cache\") pod \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") "
Jul 15 14:37:59.419649 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.419633    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tmp-dir\") pod \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") "
Jul 15 14:37:59.419773 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.419752    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb776\" (UniqueName: \"kubernetes.io/projected/79718661-d5a4-4b7f-89ad-7908bb7f0b37-kube-api-access-xb776\") pod \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") "
Jul 15 14:37:59.419853 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.419838    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "79718661-d5a4-4b7f-89ad-7908bb7f0b37" (UID: "79718661-d5a4-4b7f-89ad-7908bb7f0b37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:37:59.419912 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.419842    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-dshm\") pod \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") "
Jul 15 14:37:59.419912 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.419855    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-model-cache" (OuterVolumeSpecName: "model-cache") pod "79718661-d5a4-4b7f-89ad-7908bb7f0b37" (UID: "79718661-d5a4-4b7f-89ad-7908bb7f0b37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:37:59.419912 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.419883    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-home\") pod \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\" (UID: \"79718661-d5a4-4b7f-89ad-7908bb7f0b37\") "
Jul 15 14:37:59.420134 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.420106    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-home" (OuterVolumeSpecName: "home") pod "79718661-d5a4-4b7f-89ad-7908bb7f0b37" (UID: "79718661-d5a4-4b7f-89ad-7908bb7f0b37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:37:59.420265 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.420251    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-home\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:37:59.420323 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.420272    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-model-cache\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:37:59.420323 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.420287    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tmp-dir\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:37:59.421773 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.421749    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-dshm" (OuterVolumeSpecName: "dshm") pod "79718661-d5a4-4b7f-89ad-7908bb7f0b37" (UID: "79718661-d5a4-4b7f-89ad-7908bb7f0b37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:37:59.421866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.421794    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79718661-d5a4-4b7f-89ad-7908bb7f0b37" (UID: "79718661-d5a4-4b7f-89ad-7908bb7f0b37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:37:59.421866 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.421809    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79718661-d5a4-4b7f-89ad-7908bb7f0b37-kube-api-access-xb776" (OuterVolumeSpecName: "kube-api-access-xb776") pod "79718661-d5a4-4b7f-89ad-7908bb7f0b37" (UID: "79718661-d5a4-4b7f-89ad-7908bb7f0b37"). InnerVolumeSpecName "kube-api-access-xb776". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:37:59.521577 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.521539    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79718661-d5a4-4b7f-89ad-7908bb7f0b37-tls-certs\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:37:59.521577 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.521570    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb776\" (UniqueName: \"kubernetes.io/projected/79718661-d5a4-4b7f-89ad-7908bb7f0b37-kube-api-access-xb776\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:37:59.521577 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.521580    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79718661-d5a4-4b7f-89ad-7908bb7f0b37-dshm\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:37:59.893665 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.893628    2573 generic.go:356] "Generic (PLEG): container finished" podID="79718661-d5a4-4b7f-89ad-7908bb7f0b37" containerID="e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f" exitCode=0
Jul 15 14:37:59.893665 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.893667    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"
Jul 15 14:37:59.893862 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.893700    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr" event={"ID":"79718661-d5a4-4b7f-89ad-7908bb7f0b37","Type":"ContainerDied","Data":"e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f"}
Jul 15 14:37:59.893862 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.893742    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr" event={"ID":"79718661-d5a4-4b7f-89ad-7908bb7f0b37","Type":"ContainerDied","Data":"069460ca1d13adea9eb831573f9cf3291cc8bffc7ec534140164a6d2d5a54da4"}
Jul 15 14:37:59.893862 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.893759    2573 scope.go:117] "RemoveContainer" containerID="e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f"
Jul 15 14:37:59.903472 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.903434    2573 scope.go:117] "RemoveContainer" containerID="e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f"
Jul 15 14:37:59.903800 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:37:59.903779    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f\": container with ID starting with e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f not found: ID does not exist" containerID="e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f"
Jul 15 14:37:59.903868 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.903812    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f"} err="failed to get container status \"e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f\": rpc error: code = NotFound desc = could not find container \"e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f\": container with ID starting with e8dd570798090ac419e195616d832f03a903bdcdd9023690ce3f65fc06f3b15f not found: ID does not exist"
Jul 15 14:37:59.917096 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.917054    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"]
Jul 15 14:37:59.920960 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:37:59.920927    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0b4a59/rate-test-model-0b4a59-kserve-5c9bbd48c9-g8zsr"]
Jul 15 14:38:00.931972 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.931936    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7772-140622177816416"]
Jul 15 14:38:00.932399 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.932356    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="670486e1-86b4-41e4-bfcd-224ed244865c" containerName="test-curl-7772-140622177815552"
Jul 15 14:38:00.932399 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.932369    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="670486e1-86b4-41e4-bfcd-224ed244865c" containerName="test-curl-7772-140622177815552"
Jul 15 14:38:00.932399 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.932391    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79718661-d5a4-4b7f-89ad-7908bb7f0b37" containerName="main"
Jul 15 14:38:00.932399 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.932397    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="79718661-d5a4-4b7f-89ad-7908bb7f0b37" containerName="main"
Jul 15 14:38:00.932545 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.932460    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="79718661-d5a4-4b7f-89ad-7908bb7f0b37" containerName="main"
Jul 15 14:38:00.932545 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.932470    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="670486e1-86b4-41e4-bfcd-224ed244865c" containerName="test-curl-7772-140622177815552"
Jul 15 14:38:00.937030 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.937003    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177816416"
Jul 15 14:38:00.939956 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.939930    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5rt4r\""
Jul 15 14:38:00.940084 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:00.940025    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177816416"]
Jul 15 14:38:01.036240 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:01.036199    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmdzx\" (UniqueName: \"kubernetes.io/projected/7b95c4f9-b48c-4716-a287-2685d0e964d1-kube-api-access-dmdzx\") pod \"test-curl-7772-140622177816416\" (UID: \"7b95c4f9-b48c-4716-a287-2685d0e964d1\") " pod="opendatahub/test-curl-7772-140622177816416"
Jul 15 14:38:01.137592 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:01.137558    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmdzx\" (UniqueName: \"kubernetes.io/projected/7b95c4f9-b48c-4716-a287-2685d0e964d1-kube-api-access-dmdzx\") pod \"test-curl-7772-140622177816416\" (UID: \"7b95c4f9-b48c-4716-a287-2685d0e964d1\") " pod="opendatahub/test-curl-7772-140622177816416"
Jul 15 14:38:01.149031 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:01.148998    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmdzx\" (UniqueName: \"kubernetes.io/projected/7b95c4f9-b48c-4716-a287-2685d0e964d1-kube-api-access-dmdzx\") pod \"test-curl-7772-140622177816416\" (UID: \"7b95c4f9-b48c-4716-a287-2685d0e964d1\") " pod="opendatahub/test-curl-7772-140622177816416"
Jul 15 14:38:01.247279 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:01.247186    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177816416"
Jul 15 14:38:01.407354 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:01.407294    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79718661-d5a4-4b7f-89ad-7908bb7f0b37" path="/var/lib/kubelet/pods/79718661-d5a4-4b7f-89ad-7908bb7f0b37/volumes"
Jul 15 14:38:01.595219 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:01.595192    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177816416"]
Jul 15 14:38:01.597514 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:38:01.597485    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b95c4f9_b48c_4716_a287_2685d0e964d1.slice/crio-afe4bb8cc4dd122154d3949a824e25e092d600e7f83724028d74a7fe7c30cc24 WatchSource:0}: Error finding container afe4bb8cc4dd122154d3949a824e25e092d600e7f83724028d74a7fe7c30cc24: Status 404 returned error can't find the container with id afe4bb8cc4dd122154d3949a824e25e092d600e7f83724028d74a7fe7c30cc24
Jul 15 14:38:01.907762 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:01.907677    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177816416" event={"ID":"7b95c4f9-b48c-4716-a287-2685d0e964d1","Type":"ContainerStarted","Data":"afe4bb8cc4dd122154d3949a824e25e092d600e7f83724028d74a7fe7c30cc24"}
Jul 15 14:38:02.915180 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:02.915142    2573 generic.go:356] "Generic (PLEG): container finished" podID="7b95c4f9-b48c-4716-a287-2685d0e964d1" containerID="c4f97c660b56d96b3288c4b7d3e2b9468419c894f61a1e5e3bb764b083d41bb8" exitCode=0
Jul 15 14:38:02.915635 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:02.915224    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177816416" event={"ID":"7b95c4f9-b48c-4716-a287-2685d0e964d1","Type":"ContainerDied","Data":"c4f97c660b56d96b3288c4b7d3e2b9468419c894f61a1e5e3bb764b083d41bb8"}
Jul 15 14:38:04.069017 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.068985    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177816416"
Jul 15 14:38:04.092050 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.092015    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7772-140622177816416_7b95c4f9-b48c-4716-a287-2685d0e964d1/test-curl-7772-140622177816416/0.log"
Jul 15 14:38:04.114507 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.114470    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7772-140622177816416"]
Jul 15 14:38:04.120654 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.120624    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7772-140622177816416"]
Jul 15 14:38:04.166435 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.166403    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmdzx\" (UniqueName: \"kubernetes.io/projected/7b95c4f9-b48c-4716-a287-2685d0e964d1-kube-api-access-dmdzx\") pod \"7b95c4f9-b48c-4716-a287-2685d0e964d1\" (UID: \"7b95c4f9-b48c-4716-a287-2685d0e964d1\") "
Jul 15 14:38:04.168504 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.168480    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b95c4f9-b48c-4716-a287-2685d0e964d1-kube-api-access-dmdzx" (OuterVolumeSpecName: "kube-api-access-dmdzx") pod "7b95c4f9-b48c-4716-a287-2685d0e964d1" (UID: "7b95c4f9-b48c-4716-a287-2685d0e964d1"). InnerVolumeSpecName "kube-api-access-dmdzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:38:04.268219 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.268174    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmdzx\" (UniqueName: \"kubernetes.io/projected/7b95c4f9-b48c-4716-a287-2685d0e964d1-kube-api-access-dmdzx\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:38:04.928212 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.928183    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177816416"
Jul 15 14:38:04.928406 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:04.928189    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afe4bb8cc4dd122154d3949a824e25e092d600e7f83724028d74a7fe7c30cc24"
Jul 15 14:38:04.938399 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:38:04.938368    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7772-140622177816416\" is forbidden: User \"system:node:ip-10-0-129-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-208.ec2.internal' and this object" podUID="7b95c4f9-b48c-4716-a287-2685d0e964d1" pod="opendatahub/test-curl-7772-140622177816416"
Jul 15 14:38:05.408116 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:05.408081    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b95c4f9-b48c-4716-a287-2685d0e964d1" path="/var/lib/kubelet/pods/7b95c4f9-b48c-4716-a287-2685d0e964d1/volumes"
Jul 15 14:38:14.984822 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:14.984779    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"]
Jul 15 14:38:14.985444 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:14.985051    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8" podUID="f484c474-e9fd-4788-9ccc-253cf8c3af1b" containerName="maas-api" containerID="cri-o://962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66" gracePeriod=30
Jul 15 14:38:15.441352 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.441312    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:38:15.479799 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.479767    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f484c474-e9fd-4788-9ccc-253cf8c3af1b-openshift-service-ca\") pod \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") "
Jul 15 14:38:15.479983 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.479846    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99j7k\" (UniqueName: \"kubernetes.io/projected/f484c474-e9fd-4788-9ccc-253cf8c3af1b-kube-api-access-99j7k\") pod \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") "
Jul 15 14:38:15.479983 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.479932    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls\") pod \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\" (UID: \"f484c474-e9fd-4788-9ccc-253cf8c3af1b\") "
Jul 15 14:38:15.480281 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.480239    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f484c474-e9fd-4788-9ccc-253cf8c3af1b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f484c474-e9fd-4788-9ccc-253cf8c3af1b" (UID: "f484c474-e9fd-4788-9ccc-253cf8c3af1b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:38:15.482019 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.481991    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f484c474-e9fd-4788-9ccc-253cf8c3af1b-kube-api-access-99j7k" (OuterVolumeSpecName: "kube-api-access-99j7k") pod "f484c474-e9fd-4788-9ccc-253cf8c3af1b" (UID: "f484c474-e9fd-4788-9ccc-253cf8c3af1b"). InnerVolumeSpecName "kube-api-access-99j7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:38:15.482019 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.482005    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f484c474-e9fd-4788-9ccc-253cf8c3af1b" (UID: "f484c474-e9fd-4788-9ccc-253cf8c3af1b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:38:15.581249 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.581208    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f484c474-e9fd-4788-9ccc-253cf8c3af1b-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:38:15.581249 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.581242    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99j7k\" (UniqueName: \"kubernetes.io/projected/f484c474-e9fd-4788-9ccc-253cf8c3af1b-kube-api-access-99j7k\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:38:15.581249 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.581258    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f484c474-e9fd-4788-9ccc-253cf8c3af1b-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:38:15.993524 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.993432    2573 generic.go:356] "Generic (PLEG): container finished" podID="f484c474-e9fd-4788-9ccc-253cf8c3af1b" containerID="962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66" exitCode=0
Jul 15 14:38:15.993524 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.993479    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"
Jul 15 14:38:15.993524 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.993508    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8" event={"ID":"f484c474-e9fd-4788-9ccc-253cf8c3af1b","Type":"ContainerDied","Data":"962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66"}
Jul 15 14:38:15.994009 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.993565    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8" event={"ID":"f484c474-e9fd-4788-9ccc-253cf8c3af1b","Type":"ContainerDied","Data":"81cfc92107514e5b4964e711ff5ae5fabdb9579ea6564743b568225345afa1b6"}
Jul 15 14:38:15.994009 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:15.993593    2573 scope.go:117] "RemoveContainer" containerID="962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66"
Jul 15 14:38:16.003134 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:16.003111    2573 scope.go:117] "RemoveContainer" containerID="962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66"
Jul 15 14:38:16.003466 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:38:16.003429    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66\": container with ID starting with 962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66 not found: ID does not exist" containerID="962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66"
Jul 15 14:38:16.003551 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:16.003473    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66"} err="failed to get container status \"962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66\": rpc error: code = NotFound desc = could not find container \"962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66\": container with ID starting with 962664c6093073c1d9dfccb67d83a6dd669e7269425ed4a041d770628fde7f66 not found: ID does not exist"
Jul 15 14:38:16.016885 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:16.016858    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"]
Jul 15 14:38:16.020759 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:16.020739    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0b4a59-5fc7fc8d5f-zc2c8"]
Jul 15 14:38:17.407293 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:17.407251    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f484c474-e9fd-4788-9ccc-253cf8c3af1b" path="/var/lib/kubelet/pods/f484c474-e9fd-4788-9ccc-253cf8c3af1b/volumes"
Jul 15 14:38:41.147230 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.147193    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"]
Jul 15 14:38:41.147650 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.147418    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28" podUID="e19b273f-cbbf-4170-9a08-31c1d478e70f" containerName="maas-api" containerID="cri-o://d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00" gracePeriod=30
Jul 15 14:38:41.397707 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.397627    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:38:41.440615 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.440560    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls\") pod \"e19b273f-cbbf-4170-9a08-31c1d478e70f\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") "
Jul 15 14:38:41.440800 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.440650    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e19b273f-cbbf-4170-9a08-31c1d478e70f-openshift-service-ca\") pod \"e19b273f-cbbf-4170-9a08-31c1d478e70f\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") "
Jul 15 14:38:41.440800 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.440686    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z96k\" (UniqueName: \"kubernetes.io/projected/e19b273f-cbbf-4170-9a08-31c1d478e70f-kube-api-access-7z96k\") pod \"e19b273f-cbbf-4170-9a08-31c1d478e70f\" (UID: \"e19b273f-cbbf-4170-9a08-31c1d478e70f\") "
Jul 15 14:38:41.441026 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.440988    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e19b273f-cbbf-4170-9a08-31c1d478e70f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e19b273f-cbbf-4170-9a08-31c1d478e70f" (UID: "e19b273f-cbbf-4170-9a08-31c1d478e70f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:38:41.442904 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.442875    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e19b273f-cbbf-4170-9a08-31c1d478e70f-kube-api-access-7z96k" (OuterVolumeSpecName: "kube-api-access-7z96k") pod "e19b273f-cbbf-4170-9a08-31c1d478e70f" (UID: "e19b273f-cbbf-4170-9a08-31c1d478e70f"). InnerVolumeSpecName "kube-api-access-7z96k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:38:41.443021 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.442938    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e19b273f-cbbf-4170-9a08-31c1d478e70f" (UID: "e19b273f-cbbf-4170-9a08-31c1d478e70f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:38:41.541593 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.541551    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e19b273f-cbbf-4170-9a08-31c1d478e70f-maas-api-tls\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:38:41.541593 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.541588    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e19b273f-cbbf-4170-9a08-31c1d478e70f-openshift-service-ca\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:38:41.541593 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:41.541601    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z96k\" (UniqueName: \"kubernetes.io/projected/e19b273f-cbbf-4170-9a08-31c1d478e70f-kube-api-access-7z96k\") on node \"ip-10-0-129-208.ec2.internal\" DevicePath \"\""
Jul 15 14:38:42.136599 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.136562    2573 generic.go:356] "Generic (PLEG): container finished" podID="e19b273f-cbbf-4170-9a08-31c1d478e70f" containerID="d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00" exitCode=0
Jul 15 14:38:42.136808 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.136613    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"
Jul 15 14:38:42.136808 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.136639    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28" event={"ID":"e19b273f-cbbf-4170-9a08-31c1d478e70f","Type":"ContainerDied","Data":"d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00"}
Jul 15 14:38:42.136808 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.136688    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28" event={"ID":"e19b273f-cbbf-4170-9a08-31c1d478e70f","Type":"ContainerDied","Data":"31377511b922c1a1a876d413debcc1c1f092fd276b96cedad3996d5da153fb1e"}
Jul 15 14:38:42.136808 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.136710    2573 scope.go:117] "RemoveContainer" containerID="d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00"
Jul 15 14:38:42.146248 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.146229    2573 scope.go:117] "RemoveContainer" containerID="d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00"
Jul 15 14:38:42.146582 ip-10-0-129-208 kubenswrapper[2573]: E0715 14:38:42.146560    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00\": container with ID starting with d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00 not found: ID does not exist" containerID="d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00"
Jul 15 14:38:42.146638 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.146593    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00"} err="failed to get container status \"d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00\": rpc error: code = NotFound desc = could not find container \"d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00\": container with ID starting with d668a5265481a1cf6ab4b09d7c39017b85bc00d23ed0b9920510d93f8c416e00 not found: ID does not exist"
Jul 15 14:38:42.159063 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.159032    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"]
Jul 15 14:38:42.162932 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:42.162903    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0a4dfd-698898b67d-l5b28"]
Jul 15 14:38:43.406394 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:38:43.406360    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e19b273f-cbbf-4170-9a08-31c1d478e70f" path="/var/lib/kubelet/pods/e19b273f-cbbf-4170-9a08-31c1d478e70f/volumes"
Jul 15 14:39:03.349166 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:03.349136    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58577d4d66-2l6zh_327f3e07-1ae0-49cc-8591-80bbc95622cf/copy-manifests/0.log"
Jul 15 14:39:03.356672 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:03.356634    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58577d4d66-2l6zh_327f3e07-1ae0-49cc-8591-80bbc95622cf/manager/0.log"
Jul 15 14:39:04.360044 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:04.360011    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735430-d5n45_25139253-7492-4f7f-9992-37832c969880/cleanup/0.log"
Jul 15 14:39:04.489985 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:04.489955    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-af5d844d-hpbzn_95ceb32f-0c7e-4b5a-8b77-02bc1d0301e8/revoke-keys/1.log"
Jul 15 14:39:04.627267 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:04.627185    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-dd81e643-qtxwf_ece82a83-93ad-4f7a-a7c6-e86ee1a2608a/revoke-keys/0.log"
Jul 15 14:39:04.751171 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:04.751140    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-752a1c-htbfv_f702fcb4-4043-429a-ba6d-b16b1a3f070f/revoke-keys/0.log"
Jul 15 14:39:05.004465 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:05.004383    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-55a6f882-czw7r_83db0a14-0e07-4d68-b232-3faa2dd36faf/revoke-keys/0.log"
Jul 15 14:39:05.274895 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:05.274868    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-1bc685-vbbgc_ceba5b1c-3734-4148-a96b-806b6fc9328b/revoke-keys/0.log"
Jul 15 14:39:05.418813 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:05.418781    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-30d2f55b-ggvjc_90f41902-068a-457f-a401-eb7bb64ec931/revoke-keys/0.log"
Jul 15 14:39:05.554413 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:05.554307    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-625773fa-wmkd2_71216994-5e7c-4a1c-81a0-22f149cb2cff/revoke-keys/0.log"
Jul 15 14:39:07.992611 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:07.992581    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9wwxv_555ef84b-66b5-45c3-8140-5516e9f0e1d3/kuadrant-console-plugin/0.log"
Jul 15 14:39:08.556625 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:08.556594    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vts9v_ecdefa7e-8999-4d37-9345-5c427874951b/manager/0.log"
Jul 15 14:39:09.342377 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:09.342328    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b5998d775-tz8rr_cb4d1ff8-c081-4d10-988c-4349d7a2becc/kube-auth-proxy/0.log"
Jul 15 14:39:09.601656 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:09.601580    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-ndj5q_edea8eea-8d49-410f-983a-8d17569c4c3b/payload-pre-processing/0.log"
Jul 15 14:39:10.399796 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:10.399766    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-tzh8d_eb838ea1-1b23-4696-adbd-bd72a7135e05/main/0.log"
Jul 15 14:39:10.895381 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:10.895348    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-8szbh_40f5cfb9-f05a-47af-8d60-8795a068fd23/main/0.log"
Jul 15 14:39:15.918926 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.918883    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tzfdj/must-gather-c45th"]
Jul 15 14:39:15.919510 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919490    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f484c474-e9fd-4788-9ccc-253cf8c3af1b" containerName="maas-api"
Jul 15 14:39:15.919601 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919513    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f484c474-e9fd-4788-9ccc-253cf8c3af1b" containerName="maas-api"
Jul 15 14:39:15.919601 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919525    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e19b273f-cbbf-4170-9a08-31c1d478e70f" containerName="maas-api"
Jul 15 14:39:15.919601 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919533    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e19b273f-cbbf-4170-9a08-31c1d478e70f" containerName="maas-api"
Jul 15 14:39:15.919601 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919550    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b95c4f9-b48c-4716-a287-2685d0e964d1" containerName="test-curl-7772-140622177816416"
Jul 15 14:39:15.919601 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919560    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b95c4f9-b48c-4716-a287-2685d0e964d1" containerName="test-curl-7772-140622177816416"
Jul 15 14:39:15.919851 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919671    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e19b273f-cbbf-4170-9a08-31c1d478e70f" containerName="maas-api"
Jul 15 14:39:15.919851 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919683    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f484c474-e9fd-4788-9ccc-253cf8c3af1b" containerName="maas-api"
Jul 15 14:39:15.919851 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.919695    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b95c4f9-b48c-4716-a287-2685d0e964d1" containerName="test-curl-7772-140622177816416"
Jul 15 14:39:15.922726 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.922701    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tzfdj/must-gather-c45th"
Jul 15 14:39:15.925617 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.925590    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tzfdj\"/\"kube-root-ca.crt\""
Jul 15 14:39:15.925750 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.925590    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tzfdj\"/\"default-dockercfg-v7lr9\""
Jul 15 14:39:15.925750 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.925725    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tzfdj\"/\"openshift-service-ca.crt\""
Jul 15 14:39:15.931869 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:15.931835    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tzfdj/must-gather-c45th"]
Jul 15 14:39:16.066841 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:16.066798    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbvvm\" (UniqueName: \"kubernetes.io/projected/4fb12a93-949a-47cb-ab55-321c2f6461ce-kube-api-access-sbvvm\") pod \"must-gather-c45th\" (UID: \"4fb12a93-949a-47cb-ab55-321c2f6461ce\") " pod="openshift-must-gather-tzfdj/must-gather-c45th"
Jul 15 14:39:16.067025 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:16.066883    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4fb12a93-949a-47cb-ab55-321c2f6461ce-must-gather-output\") pod \"must-gather-c45th\" (UID: \"4fb12a93-949a-47cb-ab55-321c2f6461ce\") " pod="openshift-must-gather-tzfdj/must-gather-c45th"
Jul 15 14:39:16.167762 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:16.167713    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbvvm\" (UniqueName: \"kubernetes.io/projected/4fb12a93-949a-47cb-ab55-321c2f6461ce-kube-api-access-sbvvm\") pod \"must-gather-c45th\" (UID: \"4fb12a93-949a-47cb-ab55-321c2f6461ce\") " pod="openshift-must-gather-tzfdj/must-gather-c45th"
Jul 15 14:39:16.167964 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:16.167805    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4fb12a93-949a-47cb-ab55-321c2f6461ce-must-gather-output\") pod \"must-gather-c45th\" (UID: \"4fb12a93-949a-47cb-ab55-321c2f6461ce\") " pod="openshift-must-gather-tzfdj/must-gather-c45th"
Jul 15 14:39:16.168172 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:16.168152    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4fb12a93-949a-47cb-ab55-321c2f6461ce-must-gather-output\") pod \"must-gather-c45th\" (UID: \"4fb12a93-949a-47cb-ab55-321c2f6461ce\") " pod="openshift-must-gather-tzfdj/must-gather-c45th"
Jul 15 14:39:16.176821 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:16.176753    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbvvm\" (UniqueName: \"kubernetes.io/projected/4fb12a93-949a-47cb-ab55-321c2f6461ce-kube-api-access-sbvvm\") pod \"must-gather-c45th\" (UID: \"4fb12a93-949a-47cb-ab55-321c2f6461ce\") " pod="openshift-must-gather-tzfdj/must-gather-c45th"
Jul 15 14:39:16.233669 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:16.233628    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tzfdj/must-gather-c45th"
Jul 15 14:39:16.379245 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:16.379206    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tzfdj/must-gather-c45th"]
Jul 15 14:39:16.382930 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:39:16.382901    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fb12a93_949a_47cb_ab55_321c2f6461ce.slice/crio-14b9cdecf01e133a0f13a98dc5aad2d1309c3b4f8294ca12cfbe5a170fe40138 WatchSource:0}: Error finding container 14b9cdecf01e133a0f13a98dc5aad2d1309c3b4f8294ca12cfbe5a170fe40138: Status 404 returned error can't find the container with id 14b9cdecf01e133a0f13a98dc5aad2d1309c3b4f8294ca12cfbe5a170fe40138
Jul 15 14:39:17.324737 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:17.324700    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzfdj/must-gather-c45th" event={"ID":"4fb12a93-949a-47cb-ab55-321c2f6461ce","Type":"ContainerStarted","Data":"14b9cdecf01e133a0f13a98dc5aad2d1309c3b4f8294ca12cfbe5a170fe40138"}
Jul 15 14:39:19.341271 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:19.341219    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzfdj/must-gather-c45th" event={"ID":"4fb12a93-949a-47cb-ab55-321c2f6461ce","Type":"ContainerStarted","Data":"1905d612a6d897da835a87e2e8f835ddaa6980690f51e44719b35318482a168f"}
Jul 15 14:39:19.341892 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:19.341285    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzfdj/must-gather-c45th" event={"ID":"4fb12a93-949a-47cb-ab55-321c2f6461ce","Type":"ContainerStarted","Data":"46c9de0d6c8ae19f10fb45d717f220244581cc1e5f555993bdd016fa2e84e94e"}
Jul 15 14:39:19.361683 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:19.361607    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tzfdj/must-gather-c45th" podStartSLOduration=2.4747910539999998 podStartE2EDuration="4.361582527s" totalImagesPullingTime="1.886791473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:39:19.358871556 +0000 UTC m=+2880.478677138" watchObservedRunningTime="2026-07-15 14:39:19.361582527 +0000 UTC m=+2880.481388109"
Jul 15 14:39:19.943292 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:19.943262    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qnhlv_6edabe15-150f-4758-8b05-a9c0f1b266bf/global-pull-secret-syncer/0.log"
Jul 15 14:39:20.200614 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:20.200583    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pnhlx_af8f200d-6b67-4f66-8c9a-2d9e423b9247/konnectivity-agent/0.log"
Jul 15 14:39:20.224917 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:20.224885    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-208.ec2.internal_5e182e8671520a5dbfd171d137a66aa6/haproxy/0.log"
Jul 15 14:39:25.755915 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:25.755880    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9wwxv_555ef84b-66b5-45c3-8140-5516e9f0e1d3/kuadrant-console-plugin/0.log"
Jul 15 14:39:26.131413 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:26.131372    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vts9v_ecdefa7e-8999-4d37-9345-5c427874951b/manager/0.log"
Jul 15 14:39:27.731029 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:27.730994    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_803e9481-cf13-4f7a-87e8-e6a8c93c5bc0/alertmanager/0.log"
Jul 15 14:39:27.765955 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:27.765794    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_803e9481-cf13-4f7a-87e8-e6a8c93c5bc0/config-reloader/0.log"
Jul 15 14:39:27.803632 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:27.803573    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_803e9481-cf13-4f7a-87e8-e6a8c93c5bc0/kube-rbac-proxy-web/0.log"
Jul 15 14:39:27.842566 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:27.842530    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_803e9481-cf13-4f7a-87e8-e6a8c93c5bc0/kube-rbac-proxy/0.log"
Jul 15 14:39:27.904550 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:27.904511    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_803e9481-cf13-4f7a-87e8-e6a8c93c5bc0/kube-rbac-proxy-metric/0.log"
Jul 15 14:39:27.967505 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:27.967473    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_803e9481-cf13-4f7a-87e8-e6a8c93c5bc0/prom-label-proxy/0.log"
Jul 15 14:39:28.078640 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.078609    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_803e9481-cf13-4f7a-87e8-e6a8c93c5bc0/init-config-reloader/0.log"
Jul 15 14:39:28.242515 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.242479    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-x8xjp_15e901bb-7e01-4872-b122-7782c92e4177/kube-state-metrics/0.log"
Jul 15 14:39:28.302661 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.302604    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-x8xjp_15e901bb-7e01-4872-b122-7782c92e4177/kube-rbac-proxy-main/0.log"
Jul 15 14:39:28.355595 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.355503    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-x8xjp_15e901bb-7e01-4872-b122-7782c92e4177/kube-rbac-proxy-self/0.log"
Jul 15 14:39:28.426738 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.426708    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6c9554f5c-ljcbm_29019267-0a03-47ef-937f-adf6b7a0ee97/metrics-server/0.log"
Jul 15 14:39:28.485277 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.485130    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-x6srl_0243e1da-7793-4a84-bad5-7a19983cea44/monitoring-plugin/0.log"
Jul 15 14:39:28.668973 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.668894    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lh55b_4c6a1483-128b-4350-bf4e-11782be7ae88/node-exporter/0.log"
Jul 15 14:39:28.699548 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.699504    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lh55b_4c6a1483-128b-4350-bf4e-11782be7ae88/kube-rbac-proxy/0.log"
Jul 15 14:39:28.724698 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.724670    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lh55b_4c6a1483-128b-4350-bf4e-11782be7ae88/init-textfile/0.log"
Jul 15 14:39:28.850162 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.850130    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-7ss4m_84d5d1ec-0377-44a4-a602-a27b899f346b/kube-rbac-proxy-main/0.log"
Jul 15 14:39:28.877106 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.877067    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-7ss4m_84d5d1ec-0377-44a4-a602-a27b899f346b/kube-rbac-proxy-self/0.log"
Jul 15 14:39:28.905096 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.905067    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-7ss4m_84d5d1ec-0377-44a4-a602-a27b899f346b/openshift-state-metrics/0.log"
Jul 15 14:39:28.954658 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.954559    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4e319b56-b61c-4ca4-b609-bcd7a96258ff/prometheus/0.log"
Jul 15 14:39:28.986266 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:28.986237    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4e319b56-b61c-4ca4-b609-bcd7a96258ff/config-reloader/0.log"
Jul 15 14:39:29.010216 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.010184    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4e319b56-b61c-4ca4-b609-bcd7a96258ff/thanos-sidecar/0.log"
Jul 15 14:39:29.036316 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.036272    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4e319b56-b61c-4ca4-b609-bcd7a96258ff/kube-rbac-proxy-web/0.log"
Jul 15 14:39:29.064975 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.064940    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4e319b56-b61c-4ca4-b609-bcd7a96258ff/kube-rbac-proxy/0.log"
Jul 15 14:39:29.090319 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.090292    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4e319b56-b61c-4ca4-b609-bcd7a96258ff/kube-rbac-proxy-thanos/0.log"
Jul 15 14:39:29.117555 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.117523    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4e319b56-b61c-4ca4-b609-bcd7a96258ff/init-config-reloader/0.log"
Jul 15 14:39:29.159012 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.158979    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-lflmd_3d2dfcf1-ded7-4f19-a0db-fbe26a91e480/prometheus-operator/0.log"
Jul 15 14:39:29.182546 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.182518    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-lflmd_3d2dfcf1-ded7-4f19-a0db-fbe26a91e480/kube-rbac-proxy/0.log"
Jul 15 14:39:29.210927 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.210849    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-fbz6l_743d3c92-54e4-44fd-8242-d1a75ef6b1d5/prometheus-operator-admission-webhook/0.log"
Jul 15 14:39:29.251621 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.251591    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55fcc86d98-jgmtk_ac0b44cc-0118-4bcc-8473-b3000b5e3ec4/telemeter-client/0.log"
Jul 15 14:39:29.279121 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.279089    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55fcc86d98-jgmtk_ac0b44cc-0118-4bcc-8473-b3000b5e3ec4/reload/0.log"
Jul 15 14:39:29.305298 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.305249    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55fcc86d98-jgmtk_ac0b44cc-0118-4bcc-8473-b3000b5e3ec4/kube-rbac-proxy/0.log"
Jul 15 14:39:29.340123 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.340090    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6969448c8b-z8cjs_034787fd-ba3c-4baa-be4a-8c963d565ae7/thanos-query/0.log"
Jul 15 14:39:29.366539 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.366506    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6969448c8b-z8cjs_034787fd-ba3c-4baa-be4a-8c963d565ae7/kube-rbac-proxy-web/0.log"
Jul 15 14:39:29.393075 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.393040    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6969448c8b-z8cjs_034787fd-ba3c-4baa-be4a-8c963d565ae7/kube-rbac-proxy/0.log"
Jul 15 14:39:29.417161 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.417126    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6969448c8b-z8cjs_034787fd-ba3c-4baa-be4a-8c963d565ae7/prom-label-proxy/0.log"
Jul 15 14:39:29.451865 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.451832    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6969448c8b-z8cjs_034787fd-ba3c-4baa-be4a-8c963d565ae7/kube-rbac-proxy-rules/0.log"
Jul 15 14:39:29.476558 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:29.476462    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6969448c8b-z8cjs_034787fd-ba3c-4baa-be4a-8c963d565ae7/kube-rbac-proxy-metrics/0.log"
Jul 15 14:39:31.363874 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.363833    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"]
Jul 15 14:39:31.370481 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.370448    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.382622 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.382589    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"]
Jul 15 14:39:31.435704 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.435666    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-sys\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.435893 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.435775    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-lib-modules\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.435893 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.435821    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj7xs\" (UniqueName: \"kubernetes.io/projected/246fecad-2bd3-4bab-939e-cb058e50ac02-kube-api-access-jj7xs\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.435893 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.435849    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-podres\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.435893 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.435882    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-proc\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.536961 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.536844    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-sys\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.536961 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.536951    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-lib-modules\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.545493 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.536971    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-sys\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.545493 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.537005    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj7xs\" (UniqueName: \"kubernetes.io/projected/246fecad-2bd3-4bab-939e-cb058e50ac02-kube-api-access-jj7xs\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.545493 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.537040    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-podres\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.545493 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.537071    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-proc\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.545493 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.537115    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-lib-modules\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.545493 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.537157    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-proc\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.545493 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.537206    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/246fecad-2bd3-4bab-939e-cb058e50ac02-podres\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.549065 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.549030    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj7xs\" (UniqueName: \"kubernetes.io/projected/246fecad-2bd3-4bab-939e-cb058e50ac02-kube-api-access-jj7xs\") pod \"perf-node-gather-daemonset-57l6z\" (UID: \"246fecad-2bd3-4bab-939e-cb058e50ac02\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.685639 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.685544    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:31.730419 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.730385    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-dx9lz_e7fe77c2-bfaf-4163-83b8-24a2cba8b380/download-server/0.log"
Jul 15 14:39:31.873250 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:31.873205    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"]
Jul 15 14:39:31.879458 ip-10-0-129-208 kubenswrapper[2573]: W0715 14:39:31.877716    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod246fecad_2bd3_4bab_939e_cb058e50ac02.slice/crio-4a94dad6b4520d2dbe3e61a70e203f11d3a93ff896636cd5557694d6119d8fa4 WatchSource:0}: Error finding container 4a94dad6b4520d2dbe3e61a70e203f11d3a93ff896636cd5557694d6119d8fa4: Status 404 returned error can't find the container with id 4a94dad6b4520d2dbe3e61a70e203f11d3a93ff896636cd5557694d6119d8fa4
Jul 15 14:39:32.464485 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:32.464440    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z" event={"ID":"246fecad-2bd3-4bab-939e-cb058e50ac02","Type":"ContainerStarted","Data":"20cd4ba5dab40a9afd041fa97e33ccb97ab55dd205561fa9a4205ac3edc5782f"}
Jul 15 14:39:32.464959 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:32.464516    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:32.464959 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:32.464540    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z" event={"ID":"246fecad-2bd3-4bab-939e-cb058e50ac02","Type":"ContainerStarted","Data":"4a94dad6b4520d2dbe3e61a70e203f11d3a93ff896636cd5557694d6119d8fa4"}
Jul 15 14:39:32.485011 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:32.484947    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z" podStartSLOduration=1.484921305 podStartE2EDuration="1.484921305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 14:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:39:32.483179069 +0000 UTC m=+2893.602984654" watchObservedRunningTime="2026-07-15 14:39:32.484921305 +0000 UTC m=+2893.604726887"
Jul 15 14:39:33.522375 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:33.522320    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hd5jl_ba4d24b8-6680-4674-a0a1-4cc0dc49948a/dns/0.log"
Jul 15 14:39:33.549565 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:33.549538    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hd5jl_ba4d24b8-6680-4674-a0a1-4cc0dc49948a/kube-rbac-proxy/0.log"
Jul 15 14:39:33.653141 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:33.653113    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k5mg8_118b09fc-fa2f-4f5e-9418-19ad1aabe3f1/dns-node-resolver/0.log"
Jul 15 14:39:34.359247 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:34.359217    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qfvwg_f40d9221-15b5-4836-83ad-42bdad956a48/node-ca/0.log"
Jul 15 14:39:35.548168 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:35.548136    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b5998d775-tz8rr_cb4d1ff8-c081-4d10-988c-4349d7a2becc/kube-auth-proxy/0.log"
Jul 15 14:39:35.651465 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:35.651430    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-ndj5q_edea8eea-8d49-410f-983a-8d17569c4c3b/payload-pre-processing/0.log"
Jul 15 14:39:36.470685 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:36.470647    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g2qkr_1a318d24-fe02-4cc0-a435-f54379e199cb/serve-healthcheck-canary/0.log"
Jul 15 14:39:37.276074 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:37.276026    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tx4jt_3e8fe6a3-b050-46e6-b2b6-107ab3300d83/kube-rbac-proxy/0.log"
Jul 15 14:39:37.301913 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:37.301879    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tx4jt_3e8fe6a3-b050-46e6-b2b6-107ab3300d83/exporter/0.log"
Jul 15 14:39:37.327513 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:37.327478    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tx4jt_3e8fe6a3-b050-46e6-b2b6-107ab3300d83/extractor/0.log"
Jul 15 14:39:38.480948 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:38.480916    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-57l6z"
Jul 15 14:39:39.929695 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:39.929665    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58577d4d66-2l6zh_327f3e07-1ae0-49cc-8591-80bbc95622cf/manager/0.log"
Jul 15 14:39:39.956463 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:39.956433    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58577d4d66-2l6zh_327f3e07-1ae0-49cc-8591-80bbc95622cf/copy-manifests/0.log"
Jul 15 14:39:41.574296 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:41.574262    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b5998d775-tz8rr_cb4d1ff8-c081-4d10-988c-4349d7a2becc/kube-auth-proxy/0.log"
Jul 15 14:39:41.632072 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:41.632029    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6b767cd864-c575r_3c9543b1-2ae2-4b78-80d7-b90d9d68b802/manager/0.log"
Jul 15 14:39:41.654461 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:41.654425    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-ndj5q_edea8eea-8d49-410f-983a-8d17569c4c3b/payload-pre-processing/0.log"
Jul 15 14:39:43.104581 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:43.104550    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b5998d775-tz8rr_cb4d1ff8-c081-4d10-988c-4349d7a2becc/kube-auth-proxy/0.log"
Jul 15 14:39:43.184409 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:43.184300    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-ndj5q_edea8eea-8d49-410f-983a-8d17569c4c3b/payload-pre-processing/0.log"
Jul 15 14:39:48.683814 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:48.683786    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vffsz_b6e24913-c8a6-4c95-9a1c-e8755606b7ad/kube-multus-additional-cni-plugins/0.log"
Jul 15 14:39:48.707909 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:48.707874    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vffsz_b6e24913-c8a6-4c95-9a1c-e8755606b7ad/egress-router-binary-copy/0.log"
Jul 15 14:39:48.730541 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:48.730513    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vffsz_b6e24913-c8a6-4c95-9a1c-e8755606b7ad/cni-plugins/0.log"
Jul 15 14:39:48.752981 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:48.752950    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vffsz_b6e24913-c8a6-4c95-9a1c-e8755606b7ad/bond-cni-plugin/0.log"
Jul 15 14:39:48.775765 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:48.775738    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vffsz_b6e24913-c8a6-4c95-9a1c-e8755606b7ad/routeoverride-cni/0.log"
Jul 15 14:39:48.798700 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:48.798673    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vffsz_b6e24913-c8a6-4c95-9a1c-e8755606b7ad/whereabouts-cni-bincopy/0.log"
Jul 15 14:39:48.821352 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:48.821316    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vffsz_b6e24913-c8a6-4c95-9a1c-e8755606b7ad/whereabouts-cni/0.log"
Jul 15 14:39:49.047758 ip-10-0-129-208 kubenswrapper[2573]: I0715 14:39:49.047722    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c5qpr_4d016d94-23a8-4b4f-91f8-acd413e4558e/kube-multus/0.log"