Jul 12 15:09:41.489290 ip-10-0-142-238 systemd[1]: Starting Kubernetes Kubelet...
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.961484    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966512    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966526    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966530    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966532    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966536    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966539    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966542    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 15:09:41.969720 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966545    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966548    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966551    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966555    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966560    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966563    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966566    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966569    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966573    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966576    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966579    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966582    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966591    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966594    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966597    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966599    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966602    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966605    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966608    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 15:09:41.972239 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966617    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966622    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966626    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966629    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966632    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966634    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966637    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966640    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966642    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966645    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966647    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966650    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966654    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966656    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966660    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966662    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966665    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966668    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966670    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966673    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 15:09:42.128900 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966675    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966678    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966681    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966684    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966686    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966689    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966692    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966694    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966697    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966699    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966702    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966705    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966707    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966711    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966714    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966717    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966719    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966722    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966725    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 15:09:42.140735 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966728    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 15:09:42.034482 ip-10-0-142-238 systemd[1]: Started Kubernetes Kubelet.
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966731    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966734    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966737    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966739    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966742    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966744    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966747    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966750    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966752    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966755    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966757    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966760    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966763    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966765    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966768    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966772    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966775    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966778    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 15:09:42.141689 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966781    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966783    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966786    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966789    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966792    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966794    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966797    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966799    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966813    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966815    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966818    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966821    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966825    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966827    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.966830    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967608    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967615    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967618    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967621    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967624    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 15:09:42.142734 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967627    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967630    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967633    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967635    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967638    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967640    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967643    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967646    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967649    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967651    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967654    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967656    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967659    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967661    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967664    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967667    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967670    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967672    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967675    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967678    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 15:09:42.144599 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967680    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967682    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967686    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967690    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967695    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967699    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967702    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967704    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967707    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967710    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967712    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967715    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967717    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967721    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967724    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967726    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967729    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967731    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967733    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 15:09:42.146486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967736    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967739    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967742    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967745    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967748    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967750    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967753    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967755    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967761    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967765    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967768    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967771    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967774    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967777    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967782    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967785    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967788    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967791    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967794    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 15:09:42.147527 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967797    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967800    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967816    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967819    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967822    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967825    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967828    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967831    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967835    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967839    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967842    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967844    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967848    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967850    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967853    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967855    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967858    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967861    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967863    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 15:09:42.149535 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967866    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967868    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967871    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967873    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967876    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967878    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967885    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967887    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967890    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967894    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967898    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967903    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967908    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967911    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967914    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967916    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.967919    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968337    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968345    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968350    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 15:09:42.151852 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968354    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968368    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968373    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968377    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968381    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968385    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968391    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968396    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968400    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968404    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968408    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968412    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968416    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968420    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968424    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968428    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968434    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968439    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968445    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 15:09:42.153634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968450    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968455    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968460    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968465    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968471    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968476    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968481    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968488    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968492    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968496    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968501    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968504    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968507    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968510    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968513    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968517    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968520    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968522    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968525    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968528    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 15:09:42.159665 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968531    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968534    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968536    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968539    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968541    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968544    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968547    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968549    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968552    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968555    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968557    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968562    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968565    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968568    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968571    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968573    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968578    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968580    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968583    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968586    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968588    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 15:09:42.160486 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968591    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968594    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968596    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968599    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968601    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968604    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968606    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968609    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968612    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968615    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968617    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968620    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968622    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968625    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968628    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968630    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968633    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968636    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968638    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968641    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 15:09:42.162721 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968643    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968646    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968650    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968654    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968656    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968659    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968661    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968666    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968668    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968671    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968673    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968676    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968680    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968683    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968686    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.968689    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968785    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968793    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968825    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968832    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968843    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 12 15:09:42.163444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968848    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968854    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968858    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968862    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968865    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968869    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968873    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968876    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968879    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968882    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968886    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968888    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968892    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968898    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968901    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968904    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968907    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968910    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968917    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968921    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968926    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968930    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968933    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968936    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 12 15:09:42.164434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968939    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968942    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968946    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968950    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968953    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968956    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968959    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968962    2578 flags.go:64] FLAG: --enable-server="true"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968965    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968970    2578 flags.go:64] FLAG: --event-burst="100"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968973    2578 flags.go:64] FLAG: --event-qps="50"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968976    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968979    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968982    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968986    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968989    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968992    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968995    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.968998    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969001    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969004    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969008    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969011    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969014    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969017    2578 flags.go:64] FLAG: --feature-gates=""
Jul 12 15:09:42.165586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969021    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969026    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969029    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969032    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969036    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969039    2578 flags.go:64] FLAG: --help="false"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969042    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969045    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969048    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969051    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969055    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969058    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969061    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969064    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969067    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969070    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969074    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969077    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969080    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969083    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969086    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969089    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969092    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969094    2578 flags.go:64] FLAG: --lock-file=""
Jul 12 15:09:42.166660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969097    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969100    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969104    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969109    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969113    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969116    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969120    2578 flags.go:64] FLAG: --logging-format="text"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969124    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969129    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969134    2578 flags.go:64] FLAG: --manifest-url=""
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969137    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969142    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969145    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969149    2578 flags.go:64] FLAG: --max-pods="110"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969152    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969155    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969158    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969161    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969164    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969167    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969170    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969178    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969180    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969183    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 12 15:09:42.415632 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969186    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969190    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969196    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969199    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969202    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969205    2578 flags.go:64] FLAG: --port="10250"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969208    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969211    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d4ba808e3a9b4032"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969214    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969217    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969220    2578 flags.go:64] FLAG: --register-node="true"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969223    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969226    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969230    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969233    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969236    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969240    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969245    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969248    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969251    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969254    2578 flags.go:64] FLAG: --runonce="false"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969257    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969259    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969262    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969265    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 12 15:09:42.416494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969268    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969271    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969274    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969277    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969280    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969283    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969286    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969289    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969292    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969295    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969298    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969303    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969307    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969309    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969313    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969316    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969319    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969323    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969327    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969330    2578 flags.go:64] FLAG: --v="2"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969335    2578 flags.go:64] FLAG: --version="false"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969340    2578 flags.go:64] FLAG: --vmodule=""
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969344    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.969347    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969473    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 15:09:42.417468 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969477    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969480    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969483    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969487    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969489    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969492    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969495    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969498    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969501    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969504    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969506    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969511    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969514    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969517    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969520    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969525    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969529    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969534    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969538    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 15:09:42.418143 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969541    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969544    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969547    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969550    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969552    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969555    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969558    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969560    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969564    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969567    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969569    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969572    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969576    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969578    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969581    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969583    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969586    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969590    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969593    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 15:09:42.418693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969597    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969600    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969604    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969608    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969612    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969617    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969620    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969624    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969628    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969631    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969635    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969639    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969643    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969646    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969651    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969654    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969658    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969662    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969666    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969670    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 15:09:42.419225 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969674    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969680    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969684    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969688    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969692    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969697    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969701    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969705    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969709    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969713    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969717    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969721    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969724    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969729    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969732    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969736    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969740    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969744    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969748    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969752    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 15:09:42.419784 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969756    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969759    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969763    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969767    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969773    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969778    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969783    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969787    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969791    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969795    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969799    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969820    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969825    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969830    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969833    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969837    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969840    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969845    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969848    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 15:09:42.420335 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.969852    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.970642    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.979424    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.979445    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.979466    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.979472    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979546    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979551    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979555    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979558    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979561    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979563    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979566    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979569    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 15:09:42.420887 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979571    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979574    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979576    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979579    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979583    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979587    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979590    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979593    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979595    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979598    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979601    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979603    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979606    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979609    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979612    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979614    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979617    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979620    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979623    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 15:09:42.421311 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979630    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979633    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979635    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979638    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979641    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979643    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979646    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979648    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979651    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979654    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979657    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979659    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979662    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979664    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979667    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979670    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979675    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979679    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979682    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979686    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 15:09:42.421851 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979689    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979692    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979695    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979698    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979701    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979704    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979707    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979710    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979715    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979718    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979721    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979723    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979726    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979729    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979731    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979734    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979737    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979739    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979742    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979744    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979747    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 15:09:42.422366 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979749    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979751    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979754    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979756    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979759    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979761    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979764    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979767    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979770    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979773    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979775    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979780    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979783    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979786    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979789    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979792    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979794    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979797    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979800    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 15:09:42.422973 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979830    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979833    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979841    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979845    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979847    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979850    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979852    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979855    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979858    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979860    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979863    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979866    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.979871    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979986    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979992    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 15:09:42.423474 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979995    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.979998    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980000    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980004    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980009    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980013    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980016    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980019    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980022    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980024    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980028    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980031    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980033    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980036    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980039    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980041    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980044    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980046    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980052    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980055    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 15:09:42.423939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980058    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980060    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980063    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980066    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980068    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980071    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980073    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980076    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980079    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980081    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980084    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980087    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980089    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980091    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980094    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980096    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980099    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980103    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980105    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980108    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 15:09:42.424938 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980110    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980113    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980115    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980118    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980121    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980123    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980126    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980128    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980130    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980133    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980136    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980139    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980141    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980144    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980146    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980149    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980151    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980154    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980156    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 15:09:42.425530 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980159    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980161    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980164    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980166    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980169    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980171    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980173    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980176    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980178    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980181    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980184    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980187    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980189    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980192    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980194    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980197    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980199    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980202    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980204    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980207    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 15:09:42.687498 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980210    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980212    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980214    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980218    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980221    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980223    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980225    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980228    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980230    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980235    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980238    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980241    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980243    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980246    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980249    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980252    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980256    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 15:09:42.689230 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:41.980259    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.980264    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.980386    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.983639    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.985268    2578 server.go:1019] "Starting client certificate rotation"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.985371    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.985411    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.989043    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.991000    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:41.998971    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.003759    2578 log.go:25] "Validated CRI v1 image API"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.004593    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 12 15:09:42.689737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.007299    2578 fs.go:135] Filesystem UUIDs: map[3dea99aa-fc6d-4f42-af52-fec257fb9e05:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a98ec165-2e7a-42ba-b61a-669dccd158a1:/dev/nvme0n1p4]
Jul 12 15:09:42.690179 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.007319    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 12 15:09:42.690256 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.014726    2578 manager.go:216] Machine: {Timestamp:2026-07-12 15:09:42.01369283 +0000 UTC m=+0.381293671 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099877 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208bf7e29a9f5c37f9e917cb592845 SystemUUID:ec208bf7-e29a-9f5c-37f9-e917cb592845 BootID:6ff75fc3-7f77-4d6b-86bb-0568398fa8e4 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6a:bd:3a:be:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6a:bd:3a:be:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:45:32:37:85:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 12 15:09:42.690256 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.014887    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 12 15:09:42.690256 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.014995    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 12 15:09:42.690256 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.016141    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 12 15:09:42.690256 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.016170    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-238.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 12 15:09:42.690256 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.016291    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 12 15:09:42.690256 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.016999    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.017010    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.017017    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.017895    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.019156    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.023825    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.023944    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.023974    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.023983    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.023995    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.024004    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.024013    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.024379    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.024400    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.026762    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.028678    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.028706    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030160    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030177    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030183    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030189    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030198    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030204    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030212    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030217    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030223    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030228    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030240    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030248    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030281    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.030286    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.033549    2578 server.go:1268] "Started kubelet"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.033640    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.033657    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.033744    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.035464    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-238.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.035489    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.035510    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.036661    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 12 15:09:42.690560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.038310    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.039607    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.041461    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-238.ec2.internal.18c1936347f21d56  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-238.ec2.internal,UID:ip-10-0-142-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-238.ec2.internal,},FirstTimestamp:2026-07-12 15:09:42.033522006 +0000 UTC m=+0.401122857,LastTimestamp:2026-07-12 15:09:42.033522006 +0000 UTC m=+0.401122857,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-238.ec2.internal,}"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.043063    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.043078    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.043162    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.043863    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.043878    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.043894    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044029    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044037    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044433    2578 factory.go:55] Registering systemd factory
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044447    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044840    2578 factory.go:153] Registering CRI-O factory
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044855    2578 factory.go:223] Registration of the crio container factory successfully
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044916    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044939    2578 factory.go:103] Registering Raw factory
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.044951    2578 manager.go:1196] Started watching for new ooms in manager
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.045382    2578 manager.go:349] Starting recovery of all containers
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.047989    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.048093    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.058354    2578 manager.go:354] Recovery completed
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.062364    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.063441    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jbnfh"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.064213    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.064241    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.064254    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientPID"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.064778    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.064788    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.064792    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.064818    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.065462    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 15:09:42.691627 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.065483    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.065493    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientPID"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.066234    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-238.ec2.internal.18c1936349c69505  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-238.ec2.internal,UID:ip-10-0-142-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-238.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-238.ec2.internal,},FirstTimestamp:2026-07-12 15:09:42.064223493 +0000 UTC m=+0.431824331,LastTimestamp:2026-07-12 15:09:42.064223493 +0000 UTC m=+0.431824331,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-238.ec2.internal,}"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.067201    2578 policy_none.go:49] "None policy: Start"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.067225    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.067240    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.068900    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jbnfh"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.070213    2578 policy_none.go:47] "Start"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.074694    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-238.ec2.internal.18c1936349c6ee84  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-238.ec2.internal,UID:ip-10-0-142-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-238.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-238.ec2.internal,},FirstTimestamp:2026-07-12 15:09:42.064246404 +0000 UTC m=+0.431847242,LastTimestamp:2026-07-12 15:09:42.064246404 +0000 UTC m=+0.431847242,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-238.ec2.internal,}"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.144056    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.147324    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.148597    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.148631    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.148654    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.148685    2578 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.150909    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.176417    2578 manager.go:337] "Starting Device Plugin manager"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.176455    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.176467    2578 server.go:87] "Starting device plugin registration server"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.176766    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.176957    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.177149    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.177208    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.177216    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.177786    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.177849    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.692581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.249304    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal"]
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.249370    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.250386    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.250429    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.250443    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientPID"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.252873    2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-238.ec2.internal\" not found" node="ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.252887    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.253028    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.253063    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.253683    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.253710    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.253722    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientPID"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.253773    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.253792    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.253819    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientPID"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.256474    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.256500    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.258452    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.258481    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.258491    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientPID"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.277264    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.278080    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.278116    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.278127    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientPID"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.278148    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.285715    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-238.ec2.internal\" not found" node="ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.286617    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.286635    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-238.ec2.internal\": node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.290338    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-238.ec2.internal\" not found" node="ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.307274    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.693389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.346065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f5dad9fbe275b20abc57935e19b2841-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal\" (UID: \"9f5dad9fbe275b20abc57935e19b2841\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.346091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f5dad9fbe275b20abc57935e19b2841-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal\" (UID: \"9f5dad9fbe275b20abc57935e19b2841\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.346108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2150346244117c78ff93c3def3065ea-config\") pod \"kube-apiserver-proxy-ip-10-0-142-238.ec2.internal\" (UID: \"a2150346244117c78ff93c3def3065ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.408055    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.446390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f5dad9fbe275b20abc57935e19b2841-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal\" (UID: \"9f5dad9fbe275b20abc57935e19b2841\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.446416    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f5dad9fbe275b20abc57935e19b2841-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal\" (UID: \"9f5dad9fbe275b20abc57935e19b2841\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.446433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2150346244117c78ff93c3def3065ea-config\") pod \"kube-apiserver-proxy-ip-10-0-142-238.ec2.internal\" (UID: \"a2150346244117c78ff93c3def3065ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.446467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2150346244117c78ff93c3def3065ea-config\") pod \"kube-apiserver-proxy-ip-10-0-142-238.ec2.internal\" (UID: \"a2150346244117c78ff93c3def3065ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.446476    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f5dad9fbe275b20abc57935e19b2841-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal\" (UID: \"9f5dad9fbe275b20abc57935e19b2841\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.446516    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f5dad9fbe275b20abc57935e19b2841-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal\" (UID: \"9f5dad9fbe275b20abc57935e19b2841\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.508733    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.590954    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.594250    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:42.694275 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.609730    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.982186 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.710674    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.982186 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.811101    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:42.982186 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.901018    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 15:09:42.982186 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:42.911446    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.983971    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.984124    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:42.984167    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:43.012403    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.075018    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 15:04:42 +0000 UTC" deadline="2028-02-16 23:16:19.254591748 +0000 UTC"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.075048    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14024h6m36.179547627s"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.106403    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.108229    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.108270    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.108284    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeHasSufficientPID"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:43.113367    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:43.260229 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:43.213674    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:43.282699    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5dad9fbe275b20abc57935e19b2841.slice/crio-11deffd933af0087cbc72a60cbf96a6a9acb5f5df39bf5d66d2b9cf2a6c47574 WatchSource:0}: Error finding container 11deffd933af0087cbc72a60cbf96a6a9acb5f5df39bf5d66d2b9cf2a6c47574: Status 404 returned error can't find the container with id 11deffd933af0087cbc72a60cbf96a6a9acb5f5df39bf5d66d2b9cf2a6c47574
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:43.283058    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2150346244117c78ff93c3def3065ea.slice/crio-b7927da574bc80a775209d3dce336e5b0da69c3ed912cbdb6a9c1a9da8f04511 WatchSource:0}: Error finding container b7927da574bc80a775209d3dce336e5b0da69c3ed912cbdb6a9c1a9da8f04511: Status 404 returned error can't find the container with id b7927da574bc80a775209d3dce336e5b0da69c3ed912cbdb6a9c1a9da8f04511
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.290338    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:43.313830    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-238.ec2.internal\" not found"
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.373220    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.445190    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.458318    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.459334    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal"
Jul 12 15:09:43.513096 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.467280    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 12 15:09:43.777363 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:43.584603    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 15:09:44.031091 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.025505    2578 apiserver.go:52] "Watching apiserver"
Jul 12 15:09:44.045178 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.044193    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 12 15:09:44.146430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.131866    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 15:09:44.146430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.135733    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 12 15:09:44.146430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.137075    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x","openshift-cluster-node-tuning-operator/tuned-b2j2j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal","openshift-multus/multus-additional-cni-plugins-gm7rc","openshift-network-diagnostics/network-check-target-wnm5s","openshift-ovn-kubernetes/ovnkube-node-rs6nm","kube-system/global-pull-secret-syncer-8tpsh","openshift-dns/node-resolver-lqhsc","openshift-image-registry/node-ca-b4mrw","openshift-multus/multus-n9rp5","openshift-multus/network-metrics-daemon-6vsjp","openshift-network-operator/iptables-alerter-67qrb","kube-system/konnectivity-agent-sw5r9"]
Jul 12 15:09:44.146430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.137483    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 12 15:09:44.146430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.137586    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-95kdd"
Jul 12 15:09:44.146430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.139913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.146430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.143204    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.146430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.145652    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.148856 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.147919    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:09:44.150776 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.150625    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nlmgc\""
Jul 12 15:09:44.151217 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.150907    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 12 15:09:44.151217 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.151113    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.152828 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.152585    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 12 15:09:44.152828 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.152800    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.152842    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153170    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-676vg\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153220    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153451    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153468    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153514    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153470    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pcrpg\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153692    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g9t68\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153762    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153692    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.153909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.153967    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:09:44.154852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.154106    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.155201    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.155459    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.155541    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.155751    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.155976    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jv554\""
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.155992    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d8f228a-3673-4078-a5b6-395be8bb8aac-iptables-alerter-script\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnkxz\" (UniqueName: \"kubernetes.io/projected/9d8f228a-3673-4078-a5b6-395be8bb8aac-kube-api-access-nnkxz\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysctl-d\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156800    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-sys\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1190c39-854f-4b29-b0de-b6b6e677834e-host\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d8f228a-3673-4078-a5b6-395be8bb8aac-host-slash\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysctl-conf\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-lib-modules\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.156982    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-var-lib-kubelet\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157010    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zskq5\" (UniqueName: \"kubernetes.io/projected/b48cfee5-5612-4d8c-b410-2d06d691e596-kube-api-access-zskq5\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.248857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157045    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1190c39-854f-4b29-b0de-b6b6e677834e-serviceca\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157092    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-modprobe-d\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-kubernetes\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157142    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-tuned\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b48cfee5-5612-4d8c-b410-2d06d691e596-tmp\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfx72\" (UniqueName: \"kubernetes.io/projected/d1190c39-854f-4b29-b0de-b6b6e677834e-kube-api-access-gfx72\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysconfig\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157261    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-systemd\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-run\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-host\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.157987    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.164897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.167110    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.167393    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6bdkd\""
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.167581    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.167616    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.167769    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.167838    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.167912    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:09:44.250119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.168555    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.168769    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.168777    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.169005    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5scgv\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.169020    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.169076    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.169356    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xns4x\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.169337    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.170086    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.172661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal" event={"ID":"a2150346244117c78ff93c3def3065ea","Type":"ContainerStarted","Data":"b7927da574bc80a775209d3dce336e5b0da69c3ed912cbdb6a9c1a9da8f04511"}
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.172734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.172796    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.195147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal" event={"ID":"9f5dad9fbe275b20abc57935e19b2841","Type":"ContainerStarted","Data":"11deffd933af0087cbc72a60cbf96a6a9acb5f5df39bf5d66d2b9cf2a6c47574"}
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.200729    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.201000    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.201193    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dl49k\""
Jul 12 15:09:44.250976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.245554    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258193    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-hosts-file\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-cni-netd\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/471cb232-a846-442d-8666-f9cafab871f0-kubelet-config\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258273    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-etc-kubernetes\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn7kg\" (UniqueName: \"kubernetes.io/projected/effea80b-8ac9-4862-86cf-e4f420b175a7-kube-api-access-sn7kg\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-lib-modules\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-cnibin\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovnkube-script-lib\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-lib-modules\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-cnibin\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258513    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-socket-dir-parent\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-modprobe-d\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.259100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b48cfee5-5612-4d8c-b410-2d06d691e596-tmp\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-modprobe-d\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ab4a2e4-6527-4d42-94c5-5c4126d3732c-konnectivity-ca\") pod \"konnectivity-agent-sw5r9\" (UID: \"3ab4a2e4-6527-4d42-94c5-5c4126d3732c\") " pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-run-netns\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258765    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-registration-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrv25\" (UniqueName: \"kubernetes.io/projected/dcc058fa-537e-49c8-a4b9-79fe8986f083-kube-api-access-zrv25\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258839    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-node-log\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258848    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258875    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovnkube-config\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-socket-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.258976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-cni-bin\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-kubelet\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnkxz\" (UniqueName: \"kubernetes.io/projected/9d8f228a-3673-4078-a5b6-395be8bb8aac-kube-api-access-nnkxz\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.260232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ab4a2e4-6527-4d42-94c5-5c4126d3732c-agent-certs\") pod \"konnectivity-agent-sw5r9\" (UID: \"3ab4a2e4-6527-4d42-94c5-5c4126d3732c\") " pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259074    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-slash\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/471cb232-a846-442d-8666-f9cafab871f0-dbus\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1190c39-854f-4b29-b0de-b6b6e677834e-host\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d8f228a-3673-4078-a5b6-395be8bb8aac-host-slash\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1190c39-854f-4b29-b0de-b6b6e677834e-host\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-var-lib-kubelet\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259262    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-os-release\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-var-lib-kubelet\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259273    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d8f228a-3673-4078-a5b6-395be8bb8aac-host-slash\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-tmp-dir\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-kubelet\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-os-release\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.262417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1190c39-854f-4b29-b0de-b6b6e677834e-serviceca\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-kubernetes\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259482    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-env-overrides\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-multus-certs\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259523    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-kubernetes\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysconfig\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-conf-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6bg8\" (UniqueName: \"kubernetes.io/projected/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-kube-api-access-q6bg8\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovn-node-metrics-cert\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysconfig\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysctl-conf\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.263354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259808    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zskq5\" (UniqueName: \"kubernetes.io/projected/b48cfee5-5612-4d8c-b410-2d06d691e596-kube-api-access-zskq5\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r4z7\" (UniqueName: \"kubernetes.io/projected/22f293eb-9f03-46d2-8812-0ab424de95e3-kube-api-access-4r4z7\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-ovn\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-device-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysctl-conf\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1190c39-854f-4b29-b0de-b6b6e677834e-serviceca\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.259931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-system-cni-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-netns\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-cni-multus\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-tuned\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-systemd-units\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260275    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-sys-fs\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfx72\" (UniqueName: \"kubernetes.io/projected/d1190c39-854f-4b29-b0de-b6b6e677834e-kube-api-access-gfx72\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-systemd\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.264518 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-host\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-systemd\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-etc-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260551    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-run-ovn-kubernetes\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-host\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d8f228a-3673-4078-a5b6-395be8bb8aac-iptables-alerter-script\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysctl-d\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-sys\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-sys\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260741    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-system-cni-dir\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-sysctl-d\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-log-socket\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-cni-bin\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.265719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260875    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmbh7\" (UniqueName: \"kubernetes.io/projected/2b65a855-a5e1-4c0a-8014-2962ae802b67-kube-api-access-tmbh7\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-cni-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-systemd\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-var-lib-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260977    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-k8s-cni-cncf-io\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.260996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-hostroot\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-run\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261073    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261112    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b48cfee5-5612-4d8c-b410-2d06d691e596-run\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261192    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncbxm\" (UniqueName: \"kubernetes.io/projected/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-kube-api-access-ncbxm\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-etc-selinux\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/effea80b-8ac9-4862-86cf-e4f420b175a7-cni-binary-copy\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.266942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-daemon-config\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.267829 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.261317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d8f228a-3673-4078-a5b6-395be8bb8aac-iptables-alerter-script\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.267829 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.262951    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b48cfee5-5612-4d8c-b410-2d06d691e596-tmp\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.267829 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.262986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b48cfee5-5612-4d8c-b410-2d06d691e596-etc-tuned\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361552    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-etc-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361639    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-run-ovn-kubernetes\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361676    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-system-cni-dir\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-log-socket\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361745    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-etc-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361754    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-cni-bin\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-system-cni-dir\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmbh7\" (UniqueName: \"kubernetes.io/projected/2b65a855-a5e1-4c0a-8014-2962ae802b67-kube-api-access-tmbh7\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361814    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-cni-bin\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-run-ovn-kubernetes\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361820    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-cni-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-systemd\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-log-socket\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361907    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-var-lib-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361944    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-var-lib-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-k8s-cni-cncf-io\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361984    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-k8s-cni-cncf-io\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361987    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-systemd\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.361992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-hostroot\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-hostroot\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362040    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-cni-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362128    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncbxm\" (UniqueName: \"kubernetes.io/projected/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-kube-api-access-ncbxm\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-etc-selinux\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.383895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/effea80b-8ac9-4862-86cf-e4f420b175a7-cni-binary-copy\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-daemon-config\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-etc-selinux\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-hosts-file\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-cni-netd\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/471cb232-a846-442d-8666-f9cafab871f0-kubelet-config\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-hosts-file\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362456    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-etc-kubernetes\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-etc-kubernetes\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362529    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-cni-netd\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn7kg\" (UniqueName: \"kubernetes.io/projected/effea80b-8ac9-4862-86cf-e4f420b175a7-kube-api-access-sn7kg\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362570    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362579    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/471cb232-a846-442d-8666-f9cafab871f0-kubelet-config\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-cnibin\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovnkube-script-lib\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.384778 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.362659    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-cnibin\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.362728    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:44.862706722 +0000 UTC m=+3.230307571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362738    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-cnibin\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-cnibin\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362830    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-daemon-config\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-socket-dir-parent\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ab4a2e4-6527-4d42-94c5-5c4126d3732c-konnectivity-ca\") pod \"konnectivity-agent-sw5r9\" (UID: \"3ab4a2e4-6527-4d42-94c5-5c4126d3732c\") " pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362955    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-run-netns\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362957    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-socket-dir-parent\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.362971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-registration-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrv25\" (UniqueName: \"kubernetes.io/projected/dcc058fa-537e-49c8-a4b9-79fe8986f083-kube-api-access-zrv25\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:44.387661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363021    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-run-netns\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-registration-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363044    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-node-log\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovnkube-config\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363101    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-socket-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-cni-bin\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363175    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-node-log\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-kubelet\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ab4a2e4-6527-4d42-94c5-5c4126d3732c-agent-certs\") pod \"konnectivity-agent-sw5r9\" (UID: \"3ab4a2e4-6527-4d42-94c5-5c4126d3732c\") " pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.363261    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363264    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovnkube-script-lib\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.363311    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret podName:471cb232-a846-442d-8666-f9cafab871f0 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:44.863293349 +0000 UTC m=+3.230894177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret") pod "global-pull-secret-syncer-8tpsh" (UID: "471cb232-a846-442d-8666-f9cafab871f0") : object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363336    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-socket-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.388911 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363353    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-cni-bin\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-slash\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/471cb232-a846-442d-8666-f9cafab871f0-dbus\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-os-release\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-kubelet\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-tmp-dir\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-kubelet\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/471cb232-a846-442d-8666-f9cafab871f0-dbus\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-slash\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363566    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-os-release\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-kubelet\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363606    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-os-release\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363597    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.390645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363607    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ab4a2e4-6527-4d42-94c5-5c4126d3732c-konnectivity-ca\") pod \"konnectivity-agent-sw5r9\" (UID: \"3ab4a2e4-6527-4d42-94c5-5c4126d3732c\") " pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22f293eb-9f03-46d2-8812-0ab424de95e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-env-overrides\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-multus-certs\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363714    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-multus-certs\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363750    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22f293eb-9f03-46d2-8812-0ab424de95e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-os-release\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363803    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-conf-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363823    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovnkube-config\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6bg8\" (UniqueName: \"kubernetes.io/projected/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-kube-api-access-q6bg8\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.391499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/effea80b-8ac9-4862-86cf-e4f420b175a7-cni-binary-copy\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-multus-conf-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-openvswitch\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovn-node-metrics-cert\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r4z7\" (UniqueName: \"kubernetes.io/projected/22f293eb-9f03-46d2-8812-0ab424de95e3-kube-api-access-4r4z7\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.363971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-ovn\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364013    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-run-ovn\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364028    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-env-overrides\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-device-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-tmp-dir\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-device-dir\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-system-cni-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-system-cni-dir\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-netns\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-cni-multus\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.404033 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364180    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-systemd-units\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.404860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364196    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-run-netns\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.404860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-sys-fs\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.404860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364229    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/effea80b-8ac9-4862-86cf-e4f420b175a7-host-var-lib-cni-multus\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.404860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364246    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-systemd-units\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.404860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.364275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b65a855-a5e1-4c0a-8014-2962ae802b67-sys-fs\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.404860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.366519    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-ovn-node-metrics-cert\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:44.404860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.366702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ab4a2e4-6527-4d42-94c5-5c4126d3732c-agent-certs\") pod \"konnectivity-agent-sw5r9\" (UID: \"3ab4a2e4-6527-4d42-94c5-5c4126d3732c\") " pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:09:44.522559 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.510305    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.868120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.868181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.868295    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.868315    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.868373    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret podName:471cb232-a846-442d-8666-f9cafab871f0 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:45.868355726 +0000 UTC m=+4.235956570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret") pod "global-pull-secret-syncer-8tpsh" (UID: "471cb232-a846-442d-8666-f9cafab871f0") : object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.868394    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:45.868383185 +0000 UTC m=+4.235984012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.901023    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.901057    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.901075    2578 projected.go:196] Error preparing data for projected volume kube-api-access-47wqt for pod openshift-network-diagnostics/network-check-target-wnm5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:44.901154    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt podName:5940a558-9302-45b2-834c-c6e2e0cc44d8 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:45.401131436 +0000 UTC m=+3.768732269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-47wqt" (UniqueName: "kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt") pod "network-check-target-wnm5s" (UID: "5940a558-9302-45b2-834c-c6e2e0cc44d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.905992    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn7kg\" (UniqueName: \"kubernetes.io/projected/effea80b-8ac9-4862-86cf-e4f420b175a7-kube-api-access-sn7kg\") pod \"multus-n9rp5\" (UID: \"effea80b-8ac9-4862-86cf-e4f420b175a7\") " pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.906060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrv25\" (UniqueName: \"kubernetes.io/projected/dcc058fa-537e-49c8-a4b9-79fe8986f083-kube-api-access-zrv25\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:44.949407 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.906757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfx72\" (UniqueName: \"kubernetes.io/projected/d1190c39-854f-4b29-b0de-b6b6e677834e-kube-api-access-gfx72\") pod \"node-ca-b4mrw\" (UID: \"d1190c39-854f-4b29-b0de-b6b6e677834e\") " pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:44.950075 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.907395    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnkxz\" (UniqueName: \"kubernetes.io/projected/9d8f228a-3673-4078-a5b6-395be8bb8aac-kube-api-access-nnkxz\") pod \"iptables-alerter-67qrb\" (UID: \"9d8f228a-3673-4078-a5b6-395be8bb8aac\") " pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:44.950075 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.907656    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6bg8\" (UniqueName: \"kubernetes.io/projected/3e91ea4b-53f0-4d60-9398-f65b111d3fa0-kube-api-access-q6bg8\") pod \"node-resolver-lqhsc\" (UID: \"3e91ea4b-53f0-4d60-9398-f65b111d3fa0\") " pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:44.950075 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.907768    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmbh7\" (UniqueName: \"kubernetes.io/projected/2b65a855-a5e1-4c0a-8014-2962ae802b67-kube-api-access-tmbh7\") pod \"aws-ebs-csi-driver-node-ck62x\" (UID: \"2b65a855-a5e1-4c0a-8014-2962ae802b67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:44.950075 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.907883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zskq5\" (UniqueName: \"kubernetes.io/projected/b48cfee5-5612-4d8c-b410-2d06d691e596-kube-api-access-zskq5\") pod \"tuned-b2j2j\" (UID: \"b48cfee5-5612-4d8c-b410-2d06d691e596\") " pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:44.950075 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.908375    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r4z7\" (UniqueName: \"kubernetes.io/projected/22f293eb-9f03-46d2-8812-0ab424de95e3-kube-api-access-4r4z7\") pod \"multus-additional-cni-plugins-gm7rc\" (UID: \"22f293eb-9f03-46d2-8812-0ab424de95e3\") " pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:44.950075 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:44.910245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncbxm\" (UniqueName: \"kubernetes.io/projected/698f9ac6-d2ce-44b0-b75a-5d2f265a3dab-kube-api-access-ncbxm\") pod \"ovnkube-node-rs6nm\" (UID: \"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab\") " pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:45.054006 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.053958    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b4mrw"
Jul 12 15:09:45.092350 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.092310    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-67qrb"
Jul 12 15:09:45.102587 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.102552    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b2j2j"
Jul 12 15:09:45.117790 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.117757    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gm7rc"
Jul 12 15:09:45.126015 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.125990    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lqhsc"
Jul 12 15:09:45.135022 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.134995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:09:45.142486 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.142435    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 15:04:44 +0000 UTC" deadline="2028-02-25 22:40:24.209842796 +0000 UTC"
Jul 12 15:09:45.142594 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.142487    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14239h30m39.067360593s"
Jul 12 15:09:45.143312 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.143288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x"
Jul 12 15:09:45.148978 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.148779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n9rp5"
Jul 12 15:09:45.473425 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.473391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:45.473623 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:45.473602    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 15:09:45.473664 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:45.473632    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 15:09:45.473664 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:45.473646    2578 projected.go:196] Error preparing data for projected volume kube-api-access-47wqt for pod openshift-network-diagnostics/network-check-target-wnm5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:45.473723 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:45.473712    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt podName:5940a558-9302-45b2-834c-c6e2e0cc44d8 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:46.473693965 +0000 UTC m=+4.841294799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-47wqt" (UniqueName: "kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt") pod "network-check-target-wnm5s" (UID: "5940a558-9302-45b2-834c-c6e2e0cc44d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:45.750037 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.749994    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b65a855_a5e1_4c0a_8014_2962ae802b67.slice/crio-20c7f6389e6b865b815657e3fcfada0ac1f9505d7999a85441f4e217ef010c94 WatchSource:0}: Error finding container 20c7f6389e6b865b815657e3fcfada0ac1f9505d7999a85441f4e217ef010c94: Status 404 returned error can't find the container with id 20c7f6389e6b865b815657e3fcfada0ac1f9505d7999a85441f4e217ef010c94
Jul 12 15:09:45.751305 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.751282    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab4a2e4_6527_4d42_94c5_5c4126d3732c.slice/crio-b5a41384f68000f9d7e4a7aeca7d84f369ae3adf657dd024e365b269c011500e WatchSource:0}: Error finding container b5a41384f68000f9d7e4a7aeca7d84f369ae3adf657dd024e365b269c011500e: Status 404 returned error can't find the container with id b5a41384f68000f9d7e4a7aeca7d84f369ae3adf657dd024e365b269c011500e
Jul 12 15:09:45.752626 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.752596    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeffea80b_8ac9_4862_86cf_e4f420b175a7.slice/crio-89b562280f5636f005f0857279573e529d0d026512365613ea5980da26b128f1 WatchSource:0}: Error finding container 89b562280f5636f005f0857279573e529d0d026512365613ea5980da26b128f1: Status 404 returned error can't find the container with id 89b562280f5636f005f0857279573e529d0d026512365613ea5980da26b128f1
Jul 12 15:09:45.755800 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.755777    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f293eb_9f03_46d2_8812_0ab424de95e3.slice/crio-230586ef9d8088b119b49cd455c6cf76caeb613bb805edfb863b91e25a863481 WatchSource:0}: Error finding container 230586ef9d8088b119b49cd455c6cf76caeb613bb805edfb863b91e25a863481: Status 404 returned error can't find the container with id 230586ef9d8088b119b49cd455c6cf76caeb613bb805edfb863b91e25a863481
Jul 12 15:09:45.779885 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.779850    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb48cfee5_5612_4d8c_b410_2d06d691e596.slice/crio-44bd5b375d4db314055492592d7c94c6547329f2f49d4dd77d514ba42821b541 WatchSource:0}: Error finding container 44bd5b375d4db314055492592d7c94c6547329f2f49d4dd77d514ba42821b541: Status 404 returned error can't find the container with id 44bd5b375d4db314055492592d7c94c6547329f2f49d4dd77d514ba42821b541
Jul 12 15:09:45.781566 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.781527    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698f9ac6_d2ce_44b0_b75a_5d2f265a3dab.slice/crio-c8605396a16f6abf427ff385c3e60e8758cee8a44f5bc83db80e6ab67a8d3aba WatchSource:0}: Error finding container c8605396a16f6abf427ff385c3e60e8758cee8a44f5bc83db80e6ab67a8d3aba: Status 404 returned error can't find the container with id c8605396a16f6abf427ff385c3e60e8758cee8a44f5bc83db80e6ab67a8d3aba
Jul 12 15:09:45.782307 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.782288    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e91ea4b_53f0_4d60_9398_f65b111d3fa0.slice/crio-79d1e0c52652efe000dfd701b5352bd3b52e184c3705779d502305d92e9f229c WatchSource:0}: Error finding container 79d1e0c52652efe000dfd701b5352bd3b52e184c3705779d502305d92e9f229c: Status 404 returned error can't find the container with id 79d1e0c52652efe000dfd701b5352bd3b52e184c3705779d502305d92e9f229c
Jul 12 15:09:45.783342 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.783311    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8f228a_3673_4078_a5b6_395be8bb8aac.slice/crio-dcbc5534963ab635e805a88116fa63db29960a7cdc8adf3f12af336bac84b8fb WatchSource:0}: Error finding container dcbc5534963ab635e805a88116fa63db29960a7cdc8adf3f12af336bac84b8fb: Status 404 returned error can't find the container with id dcbc5534963ab635e805a88116fa63db29960a7cdc8adf3f12af336bac84b8fb
Jul 12 15:09:45.784443 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:09:45.784420    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1190c39_854f_4b29_b0de_b6b6e677834e.slice/crio-f93b39edd14a8cae28e7f28831259ddcf95d5583b0addff533993a80c5c24028 WatchSource:0}: Error finding container f93b39edd14a8cae28e7f28831259ddcf95d5583b0addff533993a80c5c24028: Status 404 returned error can't find the container with id f93b39edd14a8cae28e7f28831259ddcf95d5583b0addff533993a80c5c24028
Jul 12 15:09:45.877071 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.876867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:45.877188 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:45.876998    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:45.877188 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:45.877152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:45.877268 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:45.877197    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret podName:471cb232-a846-442d-8666-f9cafab871f0 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:47.877180875 +0000 UTC m=+6.244781702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret") pod "global-pull-secret-syncer-8tpsh" (UID: "471cb232-a846-442d-8666-f9cafab871f0") : object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:45.877268 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:45.877242    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:45.877335 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:45.877285    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:47.877273497 +0000 UTC m=+6.244874324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:46.087333 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.087302    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 15:09:46.142696 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.142626    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 15:04:44 +0000 UTC" deadline="2028-04-11 07:56:13.978493956 +0000 UTC"
Jul 12 15:09:46.142696 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.142684    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15328h46m27.835814545s"
Jul 12 15:09:46.149019 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.148993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:46.149168 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.148995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:46.149168 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.149111    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:46.149168 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:46.149106    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:09:46.149284 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:46.149191    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:09:46.149284 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:46.149268    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:09:46.168414 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.168383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"c8605396a16f6abf427ff385c3e60e8758cee8a44f5bc83db80e6ab67a8d3aba"}
Jul 12 15:09:46.169588 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.169560    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b2j2j" event={"ID":"b48cfee5-5612-4d8c-b410-2d06d691e596","Type":"ContainerStarted","Data":"44bd5b375d4db314055492592d7c94c6547329f2f49d4dd77d514ba42821b541"}
Jul 12 15:09:46.170646 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.170624    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerStarted","Data":"230586ef9d8088b119b49cd455c6cf76caeb613bb805edfb863b91e25a863481"}
Jul 12 15:09:46.171680 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.171657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n9rp5" event={"ID":"effea80b-8ac9-4862-86cf-e4f420b175a7","Type":"ContainerStarted","Data":"89b562280f5636f005f0857279573e529d0d026512365613ea5980da26b128f1"}
Jul 12 15:09:46.173704 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.173679    2578 generic.go:356] "Generic (PLEG): container finished" podID="9f5dad9fbe275b20abc57935e19b2841" containerID="680b98aa0cc2995df1ba378578f48d59a2d4b960c7d5ef194340e3ace1e5dbdb" exitCode=0
Jul 12 15:09:46.173790 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.173758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal" event={"ID":"9f5dad9fbe275b20abc57935e19b2841","Type":"ContainerDied","Data":"680b98aa0cc2995df1ba378578f48d59a2d4b960c7d5ef194340e3ace1e5dbdb"}
Jul 12 15:09:46.177164 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.177138    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-67qrb" event={"ID":"9d8f228a-3673-4078-a5b6-395be8bb8aac","Type":"ContainerStarted","Data":"dcbc5534963ab635e805a88116fa63db29960a7cdc8adf3f12af336bac84b8fb"}
Jul 12 15:09:46.178678 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.178650    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lqhsc" event={"ID":"3e91ea4b-53f0-4d60-9398-f65b111d3fa0","Type":"ContainerStarted","Data":"79d1e0c52652efe000dfd701b5352bd3b52e184c3705779d502305d92e9f229c"}
Jul 12 15:09:46.179739 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.179720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sw5r9" event={"ID":"3ab4a2e4-6527-4d42-94c5-5c4126d3732c","Type":"ContainerStarted","Data":"b5a41384f68000f9d7e4a7aeca7d84f369ae3adf657dd024e365b269c011500e"}
Jul 12 15:09:46.181160 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.181136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x" event={"ID":"2b65a855-a5e1-4c0a-8014-2962ae802b67","Type":"ContainerStarted","Data":"20c7f6389e6b865b815657e3fcfada0ac1f9505d7999a85441f4e217ef010c94"}
Jul 12 15:09:46.182608 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.182588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal" event={"ID":"a2150346244117c78ff93c3def3065ea","Type":"ContainerStarted","Data":"536f44b6e3cec6f68313453dcaaf760e62bce319afacc6cc8a93f99dc38fda86"}
Jul 12 15:09:46.183678 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.183659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b4mrw" event={"ID":"d1190c39-854f-4b29-b0de-b6b6e677834e","Type":"ContainerStarted","Data":"f93b39edd14a8cae28e7f28831259ddcf95d5583b0addff533993a80c5c24028"}
Jul 12 15:09:46.207462 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.207396    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-238.ec2.internal" podStartSLOduration=3.207379358 podStartE2EDuration="3.207379358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:09:46.206889736 +0000 UTC m=+4.574490587" watchObservedRunningTime="2026-07-12 15:09:46.207379358 +0000 UTC m=+4.574980210"
Jul 12 15:09:46.481313 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:46.481277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:46.481502 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:46.481487    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 15:09:46.481565 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:46.481508    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 15:09:46.481565 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:46.481520    2578 projected.go:196] Error preparing data for projected volume kube-api-access-47wqt for pod openshift-network-diagnostics/network-check-target-wnm5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:46.481662 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:46.481575    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt podName:5940a558-9302-45b2-834c-c6e2e0cc44d8 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:48.481555172 +0000 UTC m=+6.849156020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-47wqt" (UniqueName: "kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt") pod "network-check-target-wnm5s" (UID: "5940a558-9302-45b2-834c-c6e2e0cc44d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:47.201544 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:47.201400    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal" event={"ID":"9f5dad9fbe275b20abc57935e19b2841","Type":"ContainerStarted","Data":"a6d4a9635e1b9e52db8bdebb0fa1ce76ac1ec3a9507851ccd9d76b4925b0cf74"}
Jul 12 15:09:47.217740 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:47.216957    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-238.ec2.internal" podStartSLOduration=4.216935034 podStartE2EDuration="4.216935034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:09:47.215359106 +0000 UTC m=+5.582959954" watchObservedRunningTime="2026-07-12 15:09:47.216935034 +0000 UTC m=+5.584535882"
Jul 12 15:09:47.894676 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:47.894636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:47.894880 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:47.894695    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:47.894880 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:47.894856    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:47.895016 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:47.894915    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret podName:471cb232-a846-442d-8666-f9cafab871f0 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:51.89489776 +0000 UTC m=+10.262498589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret") pod "global-pull-secret-syncer-8tpsh" (UID: "471cb232-a846-442d-8666-f9cafab871f0") : object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:47.895315 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:47.895296    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:47.895392 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:47.895351    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:51.89533494 +0000 UTC m=+10.262935771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:48.151362 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:48.151271    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:48.151541 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:48.151413    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:09:48.151818 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:48.151788    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:48.151926 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:48.151906    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:09:48.151996 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:48.151964    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:48.152062 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:48.152033    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:09:48.498879 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:48.498371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:48.498879 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:48.498494    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 15:09:48.498879 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:48.498510    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 15:09:48.498879 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:48.498518    2578 projected.go:196] Error preparing data for projected volume kube-api-access-47wqt for pod openshift-network-diagnostics/network-check-target-wnm5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:48.498879 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:48.498559    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt podName:5940a558-9302-45b2-834c-c6e2e0cc44d8 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:52.498546763 +0000 UTC m=+10.866147588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-47wqt" (UniqueName: "kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt") pod "network-check-target-wnm5s" (UID: "5940a558-9302-45b2-834c-c6e2e0cc44d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:50.149387 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:50.149354    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:50.149387 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:50.149372    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:50.149885 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:50.149491    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:09:50.149885 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:50.149594    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:09:50.149885 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:50.149656    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:50.149885 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:50.149728    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:09:51.930168 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:51.930132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:51.930573 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:51.930181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:51.930573 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:51.930297    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:51.930573 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:51.930301    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:51.930573 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:51.930359    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret podName:471cb232-a846-442d-8666-f9cafab871f0 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:59.930339451 +0000 UTC m=+18.297940276 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret") pod "global-pull-secret-syncer-8tpsh" (UID: "471cb232-a846-442d-8666-f9cafab871f0") : object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:51.930573 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:51.930420    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:09:59.930408087 +0000 UTC m=+18.298008919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:52.150521 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:52.150484    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:52.150675 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:52.150599    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:09:52.151055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:52.151004    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:52.151173 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:52.151099    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:09:52.151173 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:52.151156    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:52.151277 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:52.151234    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:09:52.536257 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:52.535440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:52.536257 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:52.535720    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 15:09:52.536257 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:52.535748    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 15:09:52.536257 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:52.535763    2578 projected.go:196] Error preparing data for projected volume kube-api-access-47wqt for pod openshift-network-diagnostics/network-check-target-wnm5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:52.536257 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:52.535854    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt podName:5940a558-9302-45b2-834c-c6e2e0cc44d8 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:00.53583309 +0000 UTC m=+18.903433930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-47wqt" (UniqueName: "kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt") pod "network-check-target-wnm5s" (UID: "5940a558-9302-45b2-834c-c6e2e0cc44d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:09:54.150887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:54.150149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:54.150887 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:54.150312    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:09:54.150887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:54.150515    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:54.150887 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:54.150742    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:09:54.150887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:54.150837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:54.151470 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:54.150942    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:09:56.149357 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:56.149321    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:56.149822 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:56.149451    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:09:56.149880 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:56.149852    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:56.149973 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:56.149939    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:09:56.150103 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:56.150008    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:56.150103 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:56.150090    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:09:58.148936 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:58.148895    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:58.148936 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:58.148929    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:09:58.148936 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:58.148895    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:58.149536 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:58.149040    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:09:58.149536 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:58.149078    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:09:58.149536 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:58.149197    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:09:59.990248 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:59.990194    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:09:59.990248 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:09:59.990252    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:09:59.990800 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:59.990362    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:09:59.990800 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:59.990383    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:59.990800 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:59.990444    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret podName:471cb232-a846-442d-8666-f9cafab871f0 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:15.990423724 +0000 UTC m=+34.358024569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret") pod "global-pull-secret-syncer-8tpsh" (UID: "471cb232-a846-442d-8666-f9cafab871f0") : object "kube-system"/"original-pull-secret" not registered
Jul 12 15:09:59.990800 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:09:59.990461    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:15.990452521 +0000 UTC m=+34.358053559 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:10:00.149282 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:00.149242    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:00.149465 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:00.149243    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:00.149465 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:00.149358    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:00.149465 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:00.149245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:00.149608 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:00.149494    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:00.149608 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:00.149581    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:00.594187 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:00.594145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:00.594332 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:00.594293    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 15:10:00.594332 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:00.594315    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 15:10:00.594332 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:00.594325    2578 projected.go:196] Error preparing data for projected volume kube-api-access-47wqt for pod openshift-network-diagnostics/network-check-target-wnm5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:10:00.594432 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:00.594376    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt podName:5940a558-9302-45b2-834c-c6e2e0cc44d8 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:16.59436074 +0000 UTC m=+34.961961567 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-47wqt" (UniqueName: "kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt") pod "network-check-target-wnm5s" (UID: "5940a558-9302-45b2-834c-c6e2e0cc44d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:10:02.150557 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:02.150521    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:02.151008 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:02.150608    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:02.151008 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:02.150703    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:02.151008 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:02.150838    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:02.151008 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:02.150888    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:02.151150 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:02.151022    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:04.149152 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:04.149107    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:04.149702 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:04.149107    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:04.149702 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:04.149252    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:04.149702 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:04.149266    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:04.149702 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:04.149366    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:04.149702 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:04.149449    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:05.242228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.241988    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b4mrw" event={"ID":"d1190c39-854f-4b29-b0de-b6b6e677834e","Type":"ContainerStarted","Data":"81cf4175a86e59b6b323a0f82a8c006b1cb912543d1b3721fa0bbf09ded37db2"}
Jul 12 15:10:05.245225 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.244780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"aab18125a52f0cca7c618905512f3363a0e3d62c0823d87d1a382b34e96d2e58"}
Jul 12 15:10:05.246855 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.246780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b2j2j" event={"ID":"b48cfee5-5612-4d8c-b410-2d06d691e596","Type":"ContainerStarted","Data":"6ff6d21c7dc82cb6fd0d3e4706b050d3edace214238e106413ee74cdc25e3396"}
Jul 12 15:10:05.248261 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.248233    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerStarted","Data":"f0af3d221b7534e93402350918ac6f6edba49dd6abc6c3ddfa718051404e5be5"}
Jul 12 15:10:05.249969 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.249943    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n9rp5" event={"ID":"effea80b-8ac9-4862-86cf-e4f420b175a7","Type":"ContainerStarted","Data":"f9d4efb2da558d5e7a455d046a1305411a4d4c42aecdd241b4dcb8a37d660257"}
Jul 12 15:10:05.254206 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.254035    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lqhsc" event={"ID":"3e91ea4b-53f0-4d60-9398-f65b111d3fa0","Type":"ContainerStarted","Data":"04168b746205042940d457deac62fa29847d3adc6570be8712e0296851161ba3"}
Jul 12 15:10:05.257898 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.257190    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-b4mrw" podStartSLOduration=4.068629687 podStartE2EDuration="23.257171575s" totalImagesPullingTime="19.188541888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:05.256709824 +0000 UTC m=+23.624310670" watchObservedRunningTime="2026-07-12 15:10:05.257171575 +0000 UTC m=+23.624772424"
Jul 12 15:10:05.259419 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.258472    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sw5r9" event={"ID":"3ab4a2e4-6527-4d42-94c5-5c4126d3732c","Type":"ContainerStarted","Data":"4d74bf670e4bd26ba7465586821dec4241f3de3913b4a9ba340f703d7a6fd256"}
Jul 12 15:10:05.261858 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.261832    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x" event={"ID":"2b65a855-a5e1-4c0a-8014-2962ae802b67","Type":"ContainerStarted","Data":"ad28160c8cbed1229f2a5d1768c32a2e1ce00ebda82d8f6332d9787854411c81"}
Jul 12 15:10:05.261935 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.261896    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:10:05.276380 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.276327    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b2j2j" podStartSLOduration=4.037343079 podStartE2EDuration="23.276309217s" totalImagesPullingTime="19.238966138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:05.276270809 +0000 UTC m=+23.643871657" watchObservedRunningTime="2026-07-12 15:10:05.276309217 +0000 UTC m=+23.643910064"
Jul 12 15:10:05.323759 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.323565    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lqhsc" podStartSLOduration=4.119881043 podStartE2EDuration="23.323549249s" totalImagesPullingTime="19.203668206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:05.32303108 +0000 UTC m=+23.690631935" watchObservedRunningTime="2026-07-12 15:10:05.323549249 +0000 UTC m=+23.691150093"
Jul 12 15:10:05.344514 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.344455    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n9rp5" podStartSLOduration=4.073526317 podStartE2EDuration="23.344433124s" totalImagesPullingTime="19.270906807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:05.343645398 +0000 UTC m=+23.711246257" watchObservedRunningTime="2026-07-12 15:10:05.344433124 +0000 UTC m=+23.712033987"
Jul 12 15:10:05.359157 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:05.359112    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sw5r9" podStartSLOduration=4.127351548 podStartE2EDuration="23.359095561s" totalImagesPullingTime="19.231744013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:05.359022268 +0000 UTC m=+23.726623115" watchObservedRunningTime="2026-07-12 15:10:05.359095561 +0000 UTC m=+23.726696407"
Jul 12 15:10:06.149574 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.149537    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:06.149909 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.149644    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:06.149909 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:06.149743    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:06.149909 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:06.149644    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:06.149909 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.149650    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:06.149909 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:06.149842    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:06.266285 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.266252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"466c21ace0e5133fbdea4fda4c84b73b87537e5d206d34fd4380a99f9b352668"}
Jul 12 15:10:06.266893 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.266334    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"79da4cd887bf862222a92ce0c13109c8c894d381624f47b7d64d3d5d38bdf2ed"}
Jul 12 15:10:06.266893 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.266378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"d156bdd20c1be2a4753bcf8ca17fd263ef38307fe486fc546dbb0b65aff2ecdb"}
Jul 12 15:10:06.266893 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.266416    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"1cbd1c05a5f0ee4766209ce2b7961a90518e7caf08bd2878f2ebc90ef53bb6db"}
Jul 12 15:10:06.266893 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.266455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"3ccb2c214f33656478541d6c136d8feda80a1ed54025155bbe70d8a67c96df16"}
Jul 12 15:10:06.267511 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.267489    2578 generic.go:356] "Generic (PLEG): container finished" podID="22f293eb-9f03-46d2-8812-0ab424de95e3" containerID="f0af3d221b7534e93402350918ac6f6edba49dd6abc6c3ddfa718051404e5be5" exitCode=0
Jul 12 15:10:06.267580 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.267566    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerDied","Data":"f0af3d221b7534e93402350918ac6f6edba49dd6abc6c3ddfa718051404e5be5"}
Jul 12 15:10:06.268885 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.268855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-67qrb" event={"ID":"9d8f228a-3673-4078-a5b6-395be8bb8aac","Type":"ContainerStarted","Data":"60b535595aa002f8ad5cb8bbb0f090c4e0c43701b54ef2c7e63baff84f34e8f9"}
Jul 12 15:10:06.510271 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:06.510244    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 12 15:10:07.189230 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:07.189143    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T15:10:06.510265705Z","UUID":"6b05c6a3-25b5-420f-bc1c-37080046d9c7","Handler":null,"Name":"","Endpoint":""}
Jul 12 15:10:07.197934 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:07.197908    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 12 15:10:07.198079 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:07.197947    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 12 15:10:07.273637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:07.273595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x" event={"ID":"2b65a855-a5e1-4c0a-8014-2962ae802b67","Type":"ContainerStarted","Data":"edffc95e8e86fbf0e12759a223e43dc3723e7ec04166349c7983382af9582079"}
Jul 12 15:10:08.089662 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:08.089638    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:10:08.090178 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:08.090159    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:10:08.109591 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:08.109552    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-67qrb" podStartSLOduration=6.914895149 podStartE2EDuration="26.109537055s" totalImagesPullingTime="19.194641906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:07.287568558 +0000 UTC m=+25.655169406" watchObservedRunningTime="2026-07-12 15:10:08.109537055 +0000 UTC m=+26.477137902"
Jul 12 15:10:08.149861 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:08.149616    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:08.149861 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:08.149625    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:08.149861 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:08.149639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:08.150120 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:08.149928    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:08.150120 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:08.150052    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:08.150210 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:08.150136    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:08.279861 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:08.279816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"bc4cbc95a3c9d73b9338580502638a1546890e416f5f4e63f177b5ac0b32d1fa"}
Jul 12 15:10:08.280524 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:08.280503    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sw5r9"
Jul 12 15:10:09.284775 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:09.284728    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x" event={"ID":"2b65a855-a5e1-4c0a-8014-2962ae802b67","Type":"ContainerStarted","Data":"ad9485b1e4f7a7a7bb190e19628daa88dfcc951f4cd235b4162cecbacbf02402"}
Jul 12 15:10:09.302280 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:09.302238    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ck62x" podStartSLOduration=5.091858747 podStartE2EDuration="27.30222463s" totalImagesPullingTime="22.210365883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:09.30209417 +0000 UTC m=+27.669695018" watchObservedRunningTime="2026-07-12 15:10:09.30222463 +0000 UTC m=+27.669825476"
Jul 12 15:10:10.149189 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:10.149150    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:10.149377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:10.149273    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:10.149377 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:10.149287    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:10.149493 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:10.149403    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:10.149493 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:10.149447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:10.149586 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:10.149540    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:12.149892 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.149860    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:12.150524 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:12.149959    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:12.150524 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.150032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:12.150524 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:12.150127    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:12.150524 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.150152    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:12.150524 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:12.150217    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:12.294760 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.294719    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" event={"ID":"698f9ac6-d2ce-44b0-b75a-5d2f265a3dab","Type":"ContainerStarted","Data":"b93798222016e9aeb6c816670d9489e53d75c581bb751181cb598305e35632ca"}
Jul 12 15:10:12.294996 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.294976    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:10:12.295066 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.295053    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:10:12.296526 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.296502    2578 generic.go:356] "Generic (PLEG): container finished" podID="22f293eb-9f03-46d2-8812-0ab424de95e3" containerID="3a15a8ad112ffbcf901ff30aeba805450e86d0cf69599e2f3bc94dc20627d455" exitCode=0
Jul 12 15:10:12.296621 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.296533    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerDied","Data":"3a15a8ad112ffbcf901ff30aeba805450e86d0cf69599e2f3bc94dc20627d455"}
Jul 12 15:10:12.311050 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.311026    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:10:12.323012 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:12.322962    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm" podStartSLOduration=11.007102708 podStartE2EDuration="30.322947175s" totalImagesPullingTime="19.315844467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:12.321266268 +0000 UTC m=+30.688867114" watchObservedRunningTime="2026-07-12 15:10:12.322947175 +0000 UTC m=+30.690548021"
Jul 12 15:10:13.299463 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:13.299432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:10:13.314332 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:13.314293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:10:13.500839 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:13.500789    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wnm5s"]
Jul 12 15:10:13.500995 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:13.500980    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:13.501122 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:13.501099    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:13.503800 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:13.503750    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8tpsh"]
Jul 12 15:10:13.503939 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:13.503896    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:13.504097 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:13.504073    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:13.504387 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:13.504367    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6vsjp"]
Jul 12 15:10:13.504525 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:13.504483    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:13.504632 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:13.504585    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:14.303201 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:14.303164    2578 generic.go:356] "Generic (PLEG): container finished" podID="22f293eb-9f03-46d2-8812-0ab424de95e3" containerID="21dae9ada559a745c015621a11efb7d511ce535b0ce751a247f15304d531e6e2" exitCode=0
Jul 12 15:10:14.303679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:14.303210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerDied","Data":"21dae9ada559a745c015621a11efb7d511ce535b0ce751a247f15304d531e6e2"}
Jul 12 15:10:15.149455 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:15.149204    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:15.149551 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:15.149225    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:15.149599 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:15.149566    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:15.149637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:15.149226    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:15.149680 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:15.149628    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:15.149680 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:15.149661    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:15.308249 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:15.308151    2578 generic.go:356] "Generic (PLEG): container finished" podID="22f293eb-9f03-46d2-8812-0ab424de95e3" containerID="bbdc5012765f8553b8119d301b303e830b244a54111cdd9e94e24a4734c42678" exitCode=0
Jul 12 15:10:15.308823 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:15.308240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerDied","Data":"bbdc5012765f8553b8119d301b303e830b244a54111cdd9e94e24a4734c42678"}
Jul 12 15:10:16.002691 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:16.002603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:16.002691 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:16.002653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:16.002924 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:16.002773    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 15:10:16.002924 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:16.002775    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:10:16.002924 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:16.002854    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret podName:471cb232-a846-442d-8666-f9cafab871f0 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:48.002836073 +0000 UTC m=+66.370436903 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret") pod "global-pull-secret-syncer-8tpsh" (UID: "471cb232-a846-442d-8666-f9cafab871f0") : object "kube-system"/"original-pull-secret" not registered
Jul 12 15:10:16.002924 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:16.002874    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:48.002863792 +0000 UTC m=+66.370464623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 15:10:16.607785 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:16.607749    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:16.608193 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:16.607918    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 15:10:16.608193 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:16.607939    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 15:10:16.608193 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:16.607953    2578 projected.go:196] Error preparing data for projected volume kube-api-access-47wqt for pod openshift-network-diagnostics/network-check-target-wnm5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:10:16.608193 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:16.608010    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt podName:5940a558-9302-45b2-834c-c6e2e0cc44d8 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:48.607994764 +0000 UTC m=+66.975595597 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-47wqt" (UniqueName: "kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt") pod "network-check-target-wnm5s" (UID: "5940a558-9302-45b2-834c-c6e2e0cc44d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 15:10:17.149776 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:17.149742    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:17.149776 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:17.149763    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:17.150015 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:17.149742    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:17.150015 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:17.149904    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:10:17.150015 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:17.149985    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnm5s" podUID="5940a558-9302-45b2-834c-c6e2e0cc44d8"
Jul 12 15:10:17.150136 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:17.150060    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8tpsh" podUID="471cb232-a846-442d-8666-f9cafab871f0"
Jul 12 15:10:18.848977 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.848932    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-238.ec2.internal" event="NodeReady"
Jul 12 15:10:18.849682 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.849108    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 12 15:10:18.881011 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.880979    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"]
Jul 12 15:10:18.889511 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.889480    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65b85f644b-ss8lm"]
Jul 12 15:10:18.889674 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.889600    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:18.892278 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.892250    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 12 15:10:18.892397 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.892288    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 12 15:10:18.892397 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.892304    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-t6qrp\""
Jul 12 15:10:18.893107 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.893088    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:18.895610 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.895592    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 12 15:10:18.895610 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.895602    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 12 15:10:18.895772 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.895600    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 12 15:10:18.895772 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.895623    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hsr44\""
Jul 12 15:10:18.901913 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.901890    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 12 15:10:18.903424 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.903400    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"]
Jul 12 15:10:18.904266 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.904246    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jp65g"]
Jul 12 15:10:18.908201 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.908076    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-smg4c"]
Jul 12 15:10:18.908201 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.908188    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jp65g"
Jul 12 15:10:18.910671 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.910645    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 12 15:10:18.910671 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.910665    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 12 15:10:18.910839 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.910655    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9q7ts\""
Jul 12 15:10:18.910905 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.910885    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 12 15:10:18.911449 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.911431    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65b85f644b-ss8lm"]
Jul 12 15:10:18.911549 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.911534    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:18.913918 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.913898    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 12 15:10:18.914006 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.913934    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 12 15:10:18.914226 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.914212    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-72hfp\""
Jul 12 15:10:18.916663 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.916641    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jp65g"]
Jul 12 15:10:18.917268 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:18.917245    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-smg4c"]
Jul 12 15:10:19.028008 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.027968    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-bound-sa-token\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.028184 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f904686e-b8e3-4353-af52-687df74b253a-config-volume\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.028184 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028060    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.028184 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2f2c03a-fced-4df6-842b-257fca3c1f18-nginx-conf\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:19.028338 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-image-registry-private-configuration\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.028338 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-trusted-ca\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.028338 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-installation-pull-secrets\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.028338 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj2nr\" (UniqueName: \"kubernetes.io/projected/c0a29124-e5db-412f-b3fe-5879f9d68c88-kube-api-access-pj2nr\") pod \"ingress-canary-jp65g\" (UID: \"c0a29124-e5db-412f-b3fe-5879f9d68c88\") " pod="openshift-ingress-canary/ingress-canary-jp65g"
Jul 12 15:10:19.028575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f904686e-b8e3-4353-af52-687df74b253a-tmp-dir\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.028575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ee77f29-b901-4674-a7c5-50380d1100e6-ca-trust-extracted\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.028575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028435    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bl77\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-kube-api-access-8bl77\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.028575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf8db\" (UniqueName: \"kubernetes.io/projected/f904686e-b8e3-4353-af52-687df74b253a-kube-api-access-rf8db\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.028575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.028575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0a29124-e5db-412f-b3fe-5879f9d68c88-cert\") pod \"ingress-canary-jp65g\" (UID: \"c0a29124-e5db-412f-b3fe-5879f9d68c88\") " pod="openshift-ingress-canary/ingress-canary-jp65g"
Jul 12 15:10:19.028575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:19.028881 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.028611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-certificates\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.129370 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.129275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.129370 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.129321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2f2c03a-fced-4df6-842b-257fca3c1f18-nginx-conf\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:19.129575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.129383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-image-registry-private-configuration\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.129575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.129405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-trusted-ca\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.129575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.129425    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-installation-pull-secrets\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.129575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.129449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj2nr\" (UniqueName: \"kubernetes.io/projected/c0a29124-e5db-412f-b3fe-5879f9d68c88-kube-api-access-pj2nr\") pod \"ingress-canary-jp65g\" (UID: \"c0a29124-e5db-412f-b3fe-5879f9d68c88\") " pod="openshift-ingress-canary/ingress-canary-jp65g"
Jul 12 15:10:19.129575 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.129454    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:10:19.129575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.129473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f904686e-b8e3-4353-af52-687df74b253a-tmp-dir\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.129575 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.129543    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:10:19.629521469 +0000 UTC m=+37.997122317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:10:19.129950 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.129833    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f904686e-b8e3-4353-af52-687df74b253a-tmp-dir\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.130262 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130236    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2f2c03a-fced-4df6-842b-257fca3c1f18-nginx-conf\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:19.130374 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ee77f29-b901-4674-a7c5-50380d1100e6-ca-trust-extracted\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.130374 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bl77\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-kube-api-access-8bl77\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.130374 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf8db\" (UniqueName: \"kubernetes.io/projected/f904686e-b8e3-4353-af52-687df74b253a-kube-api-access-rf8db\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.130519 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.130519 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130446    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0a29124-e5db-412f-b3fe-5879f9d68c88-cert\") pod \"ingress-canary-jp65g\" (UID: \"c0a29124-e5db-412f-b3fe-5879f9d68c88\") " pod="openshift-ingress-canary/ingress-canary-jp65g"
Jul 12 15:10:19.130519 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:19.130657 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-certificates\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.130657 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-bound-sa-token\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.130657 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f904686e-b8e3-4353-af52-687df74b253a-config-volume\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.130789 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.130709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ee77f29-b901-4674-a7c5-50380d1100e6-ca-trust-extracted\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.130858 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.130840    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:10:19.130922 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.130907    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:19.630890165 +0000 UTC m=+37.998490993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:10:19.130993 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.130979    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:10:19.131101 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.130996    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:10:19.131101 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.131033    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:19.631020047 +0000 UTC m=+37.998620872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:10:19.131329 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.131277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f904686e-b8e3-4353-af52-687df74b253a-config-volume\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.131562 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.131532    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-certificates\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.131797 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.131701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-trusted-ca\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.134438 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.134422    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0a29124-e5db-412f-b3fe-5879f9d68c88-cert\") pod \"ingress-canary-jp65g\" (UID: \"c0a29124-e5db-412f-b3fe-5879f9d68c88\") " pod="openshift-ingress-canary/ingress-canary-jp65g"
Jul 12 15:10:19.134438 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.134435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-image-registry-private-configuration\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.134565 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.134544    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-installation-pull-secrets\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.138764 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.138709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj2nr\" (UniqueName: \"kubernetes.io/projected/c0a29124-e5db-412f-b3fe-5879f9d68c88-kube-api-access-pj2nr\") pod \"ingress-canary-jp65g\" (UID: \"c0a29124-e5db-412f-b3fe-5879f9d68c88\") " pod="openshift-ingress-canary/ingress-canary-jp65g"
Jul 12 15:10:19.139214 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.139192    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bl77\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-kube-api-access-8bl77\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.139605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.139580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf8db\" (UniqueName: \"kubernetes.io/projected/f904686e-b8e3-4353-af52-687df74b253a-kube-api-access-rf8db\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.139942 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.139921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-bound-sa-token\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.149468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.149410    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:19.149567 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.149532    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:19.149567 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.149560    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:19.152071 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.152044    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 12 15:10:19.152071 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.152065    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 12 15:10:19.152217 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.152178    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 12 15:10:19.152323 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.152305    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 12 15:10:19.152392 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.152352    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sv5b2\""
Jul 12 15:10:19.152392 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.152387    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mn9r4\""
Jul 12 15:10:19.222765 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.222727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jp65g"
Jul 12 15:10:19.636161 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.636122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:19.636354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.636206    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:19.636354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.636234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:19.636354 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.636305    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:10:19.636513 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.636359    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:10:19.636513 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.636384    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:10:19.636513 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.636405    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:10:19.636513 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.636388    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:10:20.63637029 +0000 UTC m=+39.003971116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:10:19.636513 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.636484    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:20.636461929 +0000 UTC m=+39.004062761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:10:19.636513 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:19.636501    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:20.636490474 +0000 UTC m=+39.004091304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:10:19.852464 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:19.852264    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jp65g"]
Jul 12 15:10:19.908939 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:10:19.908848    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a29124_e5db_412f_b3fe_5879f9d68c88.slice/crio-6b7434bceab4bbd4f4b365e98f0c7e1bfc58500e1c07c202dd5694f0a4837398 WatchSource:0}: Error finding container 6b7434bceab4bbd4f4b365e98f0c7e1bfc58500e1c07c202dd5694f0a4837398: Status 404 returned error can't find the container with id 6b7434bceab4bbd4f4b365e98f0c7e1bfc58500e1c07c202dd5694f0a4837398
Jul 12 15:10:20.323708 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:20.323618    2578 generic.go:356] "Generic (PLEG): container finished" podID="22f293eb-9f03-46d2-8812-0ab424de95e3" containerID="bfbaed6b1a60e940ac933fd6bb0a12d86bf4bf1e8ce64b7245c0821b87b28499" exitCode=0
Jul 12 15:10:20.323708 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:20.323696    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerDied","Data":"bfbaed6b1a60e940ac933fd6bb0a12d86bf4bf1e8ce64b7245c0821b87b28499"}
Jul 12 15:10:20.324922 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:20.324896    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jp65g" event={"ID":"c0a29124-e5db-412f-b3fe-5879f9d68c88","Type":"ContainerStarted","Data":"6b7434bceab4bbd4f4b365e98f0c7e1bfc58500e1c07c202dd5694f0a4837398"}
Jul 12 15:10:20.644436 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:20.644397    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:20.644647 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:20.644491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:20.644647 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:20.644519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:20.644647 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:20.644571    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:10:20.644647 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:20.644635    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:10:20.644647 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:20.644648    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:10:22.644628181 +0000 UTC m=+41.012229011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:10:20.644920 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:20.644694    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:22.644675492 +0000 UTC m=+41.012276320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:10:20.644920 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:20.644639    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:10:20.644920 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:20.644719    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:10:20.644920 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:20.644764    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:22.644752578 +0000 UTC m=+41.012353404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:10:21.331569 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:21.331539    2578 generic.go:356] "Generic (PLEG): container finished" podID="22f293eb-9f03-46d2-8812-0ab424de95e3" containerID="b44bbb6b7473d4bdfedf6c601bccecb89b802a37bd8e1be2e332925bc6242c8c" exitCode=0
Jul 12 15:10:21.332069 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:21.331595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerDied","Data":"b44bbb6b7473d4bdfedf6c601bccecb89b802a37bd8e1be2e332925bc6242c8c"}
Jul 12 15:10:22.338846 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:22.338561    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" event={"ID":"22f293eb-9f03-46d2-8812-0ab424de95e3","Type":"ContainerStarted","Data":"984e07323c86eaf1be243e0ff843fdf64dd2dffaf73d4d830f76c68127f076bf"}
Jul 12 15:10:22.363342 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:22.363288    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gm7rc" podStartSLOduration=9.270961783 podStartE2EDuration="40.363266694s" totalImagesPullingTime="31.092304911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:22.361209156 +0000 UTC m=+40.728810005" watchObservedRunningTime="2026-07-12 15:10:22.363266694 +0000 UTC m=+40.730867542"
Jul 12 15:10:22.665503 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:22.665412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:22.665503 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:22.665455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:22.665503 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:22.665505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:22.665781 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:22.665569    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:10:22.665781 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:22.665590    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:10:22.665781 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:22.665605    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:10:22.665781 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:22.665615    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:10:22.665781 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:22.665645    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:26.665629873 +0000 UTC m=+45.033230698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:10:22.665781 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:22.665658    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:10:26.665651998 +0000 UTC m=+45.033252824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:10:22.665781 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:22.665668    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:26.665662608 +0000 UTC m=+45.033263433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:10:23.342288 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:23.342230    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jp65g" event={"ID":"c0a29124-e5db-412f-b3fe-5879f9d68c88","Type":"ContainerStarted","Data":"e60219c5bbb8e466f6c85ba2962e2bcc2c636a661a74e38d26e5ef9257eb294b"}
Jul 12 15:10:23.356879 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:23.356833    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jp65g" podStartSLOduration=2.423830604 podStartE2EDuration="5.356791085s" totalImagesPullingTime="2.932960481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:23.356426673 +0000 UTC m=+41.724027521" watchObservedRunningTime="2026-07-12 15:10:23.356791085 +0000 UTC m=+41.724391931"
Jul 12 15:10:26.697452 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:26.697401    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:26.697452 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:26.697451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:26.697990 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:26.697524    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:26.697990 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:26.697591    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:10:26.697990 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:26.697609    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:10:26.697990 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:26.697644    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:10:26.697990 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:26.697653    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:10:26.697990 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:26.697678    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:34.69765808 +0000 UTC m=+53.065258918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:10:26.697990 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:26.697696    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:10:34.697687663 +0000 UTC m=+53.065288496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:10:26.697990 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:26.697728    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:34.697708285 +0000 UTC m=+53.065309113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:10:34.759117 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:34.759070    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:34.759117 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:34.759117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:34.759553 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:34.759162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:34.759553 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:34.759236    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:10:34.759553 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:34.759263    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:10:34.759553 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:34.759272    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:10:34.759553 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:34.759246    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:10:34.759553 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:34.759325    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:50.759307939 +0000 UTC m=+69.126908764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:10:34.759553 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:34.759338    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:10:50.759332386 +0000 UTC m=+69.126933210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:10:34.759553 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:34.759349    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:10:50.759344026 +0000 UTC m=+69.126944850 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:10:45.318393 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:45.318366    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rs6nm"
Jul 12 15:10:48.057548 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.057502    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:10:48.057548 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.057565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:48.060339 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.060322    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 12 15:10:48.060397 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.060326    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 12 15:10:48.068468 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:48.068445    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 12 15:10:48.068528 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:48.068509    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:11:52.068493118 +0000 UTC m=+130.436093943 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : secret "metrics-daemon-secret" not found
Jul 12 15:10:48.072020 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.071990    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471cb232-a846-442d-8666-f9cafab871f0-original-pull-secret\") pod \"global-pull-secret-syncer-8tpsh\" (UID: \"471cb232-a846-442d-8666-f9cafab871f0\") " pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:48.260074 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.260039    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8tpsh"
Jul 12 15:10:48.381180 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.381149    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8tpsh"]
Jul 12 15:10:48.384905 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:10:48.384871    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471cb232_a846_442d_8666_f9cafab871f0.slice/crio-b856867e5963e1c6bf964ede964140892732679100e121525ce76332145a817e WatchSource:0}: Error finding container b856867e5963e1c6bf964ede964140892732679100e121525ce76332145a817e: Status 404 returned error can't find the container with id b856867e5963e1c6bf964ede964140892732679100e121525ce76332145a817e
Jul 12 15:10:48.404470 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.404433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8tpsh" event={"ID":"471cb232-a846-442d-8666-f9cafab871f0","Type":"ContainerStarted","Data":"b856867e5963e1c6bf964ede964140892732679100e121525ce76332145a817e"}
Jul 12 15:10:48.662780 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.662680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:48.665861 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.665843    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 12 15:10:48.676174 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.676153    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 12 15:10:48.686544 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.686514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47wqt\" (UniqueName: \"kubernetes.io/projected/5940a558-9302-45b2-834c-c6e2e0cc44d8-kube-api-access-47wqt\") pod \"network-check-target-wnm5s\" (UID: \"5940a558-9302-45b2-834c-c6e2e0cc44d8\") " pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:48.871115 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.871077    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sv5b2\""
Jul 12 15:10:48.878336 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:48.878296    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:49.024040 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:49.023993    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wnm5s"]
Jul 12 15:10:49.027374 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:10:49.027340    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5940a558_9302_45b2_834c_c6e2e0cc44d8.slice/crio-0130a751e745d4c8f41cf42f461f90d6012c8e60bde2cfa357d91794dd3cec4a WatchSource:0}: Error finding container 0130a751e745d4c8f41cf42f461f90d6012c8e60bde2cfa357d91794dd3cec4a: Status 404 returned error can't find the container with id 0130a751e745d4c8f41cf42f461f90d6012c8e60bde2cfa357d91794dd3cec4a
Jul 12 15:10:49.408084 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:49.408040    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wnm5s" event={"ID":"5940a558-9302-45b2-834c-c6e2e0cc44d8","Type":"ContainerStarted","Data":"0130a751e745d4c8f41cf42f461f90d6012c8e60bde2cfa357d91794dd3cec4a"}
Jul 12 15:10:50.781120 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:50.781064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:10:50.781618 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:50.781125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:10:50.781618 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:50.781195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:10:50.781618 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:50.781341    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:10:50.781618 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:50.781444    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:11:22.781424131 +0000 UTC m=+101.149024961 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:10:50.781618 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:50.781530    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:10:50.781618 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:50.781556    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:10:50.781937 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:50.781531    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:10:50.781937 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:50.781624    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:11:22.781606851 +0000 UTC m=+101.149207677 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:10:50.781937 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:10:50.781715    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:11:22.781695127 +0000 UTC m=+101.149295955 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:10:54.422553 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:54.422519    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wnm5s" event={"ID":"5940a558-9302-45b2-834c-c6e2e0cc44d8","Type":"ContainerStarted","Data":"5f00ae4c5936d98039f90f2dda87c92cb4bf3941af24c16cdbc269b61f55b2d0"}
Jul 12 15:10:54.422981 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:54.422635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:10:54.423987 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:54.423968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8tpsh" event={"ID":"471cb232-a846-442d-8666-f9cafab871f0","Type":"ContainerStarted","Data":"26621831a81ef9fd80d8a64f1f3b4682625b4885b91debce5d0d522940545ac3"}
Jul 12 15:10:54.453084 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:54.453037    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wnm5s" podStartSLOduration=67.983445945 podStartE2EDuration="1m12.453023049s" totalImagesPullingTime="4.469577104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:54.436476083 +0000 UTC m=+72.804076930" watchObservedRunningTime="2026-07-12 15:10:54.453023049 +0000 UTC m=+72.820623915"
Jul 12 15:10:54.453336 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:10:54.453299    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8tpsh" podStartSLOduration=67.341223406 podStartE2EDuration="1m12.45329201s" totalImagesPullingTime="5.112068604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:54.44952699 +0000 UTC m=+72.817127837" watchObservedRunningTime="2026-07-12 15:10:54.45329201 +0000 UTC m=+72.820892856"
Jul 12 15:11:08.853478 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.853363    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"]
Jul 12 15:11:08.857451 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.857434    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:08.860310 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.860284    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 12 15:11:08.860424 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.860321    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 12 15:11:08.861519 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.861501    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 12 15:11:08.861580 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.861501    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 12 15:11:08.865519 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.865498    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"]
Jul 12 15:11:08.911670 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.911635    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvvrm\" (UniqueName: \"kubernetes.io/projected/abbfe80f-3352-410f-8e0f-d27455ce8743-kube-api-access-zvvrm\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:08.911893 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.911705    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/abbfe80f-3352-410f-8e0f-d27455ce8743-klusterlet-config\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:08.911893 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:08.911763    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbfe80f-3352-410f-8e0f-d27455ce8743-tmp\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:09.012493 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.012446    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/abbfe80f-3352-410f-8e0f-d27455ce8743-klusterlet-config\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:09.012493 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.012496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbfe80f-3352-410f-8e0f-d27455ce8743-tmp\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:09.012636 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.012544    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvvrm\" (UniqueName: \"kubernetes.io/projected/abbfe80f-3352-410f-8e0f-d27455ce8743-kube-api-access-zvvrm\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:09.012966 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.012943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbfe80f-3352-410f-8e0f-d27455ce8743-tmp\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:09.014947 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.014923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/abbfe80f-3352-410f-8e0f-d27455ce8743-klusterlet-config\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:09.020697 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.020672    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvvrm\" (UniqueName: \"kubernetes.io/projected/abbfe80f-3352-410f-8e0f-d27455ce8743-kube-api-access-zvvrm\") pod \"klusterlet-addon-workmgr-78dffb7cbb-ppfvd\" (UID: \"abbfe80f-3352-410f-8e0f-d27455ce8743\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:09.167263 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.167172    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:09.286466 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.286432    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"]
Jul 12 15:11:09.289995 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:11:09.289972    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbfe80f_3352_410f_8e0f_d27455ce8743.slice/crio-d5da085037c04502e3bbc82ca819ff2df325423f0c36df718be69ffdc38c113c WatchSource:0}: Error finding container d5da085037c04502e3bbc82ca819ff2df325423f0c36df718be69ffdc38c113c: Status 404 returned error can't find the container with id d5da085037c04502e3bbc82ca819ff2df325423f0c36df718be69ffdc38c113c
Jul 12 15:11:09.459246 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:09.459162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd" event={"ID":"abbfe80f-3352-410f-8e0f-d27455ce8743","Type":"ContainerStarted","Data":"d5da085037c04502e3bbc82ca819ff2df325423f0c36df718be69ffdc38c113c"}
Jul 12 15:11:13.471310 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:13.471272    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd" event={"ID":"abbfe80f-3352-410f-8e0f-d27455ce8743","Type":"ContainerStarted","Data":"8c0a1a2458094d8849844925bedd22b5d688b97a587bb5656b24374a91fe2bca"}
Jul 12 15:11:13.471766 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:13.471466    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:13.473300 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:13.473276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:11:13.485695 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:13.485643    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd" podStartSLOduration=2.001838247 podStartE2EDuration="5.485626342s" totalImagesPullingTime="3.483788095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:11:13.485229168 +0000 UTC m=+91.852830038" watchObservedRunningTime="2026-07-12 15:11:13.485626342 +0000 UTC m=+91.853227191"
Jul 12 15:11:22.818962 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:22.818921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:22.818982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:22.819005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:22.819070    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:22.819106    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:22.819119    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:22.819134    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:22.819141    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:12:26.819119155 +0000 UTC m=+165.186719981 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:22.819162    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:12:26.819149399 +0000 UTC m=+165.186750236 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:11:22.819442 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:22.819177    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:12:26.819169912 +0000 UTC m=+165.186770738 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:11:25.429106 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:25.429074    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wnm5s"
Jul 12 15:11:52.131517 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:11:52.131458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:11:52.132026 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:52.131606    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 12 15:11:52.132026 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:11:52.131682    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:13:54.131664756 +0000 UTC m=+252.499265580 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : secret "metrics-daemon-secret" not found
Jul 12 15:12:21.904197 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:21.904139    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp" podUID="b2f2c03a-fced-4df6-842b-257fca3c1f18"
Jul 12 15:12:21.910300 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:21.910272    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" podUID="9ee77f29-b901-4674-a7c5-50380d1100e6"
Jul 12 15:12:21.929602 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:21.929572    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-smg4c" podUID="f904686e-b8e3-4353-af52-687df74b253a"
Jul 12 15:12:22.174548 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:22.174458    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:12:22.641816 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:12:22.641785    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:12:22.642015 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:12:22.641786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-smg4c"
Jul 12 15:12:22.642015 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:12:22.641787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:12:26.874062 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:12:26.874024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:12:26.874084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:12:26.874111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:26.874184    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:26.874198    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:26.874237    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:26.874255    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b85f644b-ss8lm: secret "image-registry-tls" not found
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:26.874257    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert podName:b2f2c03a-fced-4df6-842b-257fca3c1f18 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:28.874239572 +0000 UTC m=+287.241840407 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-mjwlp" (UID: "b2f2c03a-fced-4df6-842b-257fca3c1f18") : secret "networking-console-plugin-cert" not found
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:26.874314    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls podName:f904686e-b8e3-4353-af52-687df74b253a nodeName:}" failed. No retries permitted until 2026-07-12 15:14:28.874301065 +0000 UTC m=+287.241901889 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls") pod "dns-default-smg4c" (UID: "f904686e-b8e3-4353-af52-687df74b253a") : secret "dns-default-metrics-tls" not found
Jul 12 15:12:26.874453 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:12:26.874324    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls podName:9ee77f29-b901-4674-a7c5-50380d1100e6 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:28.874319265 +0000 UTC m=+287.241920090 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls") pod "image-registry-65b85f644b-ss8lm" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6") : secret "image-registry-tls" not found
Jul 12 15:12:36.149361 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:12:36.149250    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:13:13.472841 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:13:13.472769    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd" podUID="abbfe80f-3352-410f-8e0f-d27455ce8743" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/readyz\": dial tcp 10.132.0.10:8000: connect: connection refused"
Jul 12 15:13:13.766246 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:13:13.766167    2578 generic.go:356] "Generic (PLEG): container finished" podID="abbfe80f-3352-410f-8e0f-d27455ce8743" containerID="8c0a1a2458094d8849844925bedd22b5d688b97a587bb5656b24374a91fe2bca" exitCode=1
Jul 12 15:13:13.766246 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:13:13.766222    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd" event={"ID":"abbfe80f-3352-410f-8e0f-d27455ce8743","Type":"ContainerDied","Data":"8c0a1a2458094d8849844925bedd22b5d688b97a587bb5656b24374a91fe2bca"}
Jul 12 15:13:13.766530 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:13:13.766518    2578 scope.go:117] "RemoveContainer" containerID="8c0a1a2458094d8849844925bedd22b5d688b97a587bb5656b24374a91fe2bca"
Jul 12 15:13:14.770674 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:13:14.770639    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd" event={"ID":"abbfe80f-3352-410f-8e0f-d27455ce8743","Type":"ContainerStarted","Data":"8a1eeaaeab44f585d2c14550db558a9e5217bff5b935b9ec3ddffdfaa2e6fdeb"}
Jul 12 15:13:14.771091 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:13:14.770923    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:13:14.771599 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:13:14.771575    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78dffb7cbb-ppfvd"
Jul 12 15:13:54.189992 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:13:54.189950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:13:54.190483 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:13:54.190089    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 12 15:13:54.190483 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:13:54.190150    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs podName:dcc058fa-537e-49c8-a4b9-79fe8986f083 nodeName:}" failed. No retries permitted until 2026-07-12 15:15:56.190134769 +0000 UTC m=+374.557735597 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs") pod "network-metrics-daemon-6vsjp" (UID: "dcc058fa-537e-49c8-a4b9-79fe8986f083") : secret "metrics-daemon-secret" not found
Jul 12 15:14:04.315018 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.314985    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9"]
Jul 12 15:14:04.317914 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.317894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9"
Jul 12 15:14:04.323232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.323210    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xdsmw\""
Jul 12 15:14:04.324514 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.324494    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 12 15:14:04.324617 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.324504    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 12 15:14:04.332953 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.332929    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9"]
Jul 12 15:14:04.364181 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.364146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9m24\" (UniqueName: \"kubernetes.io/projected/cbd2cf50-f068-4c6f-8f23-a254a4515790-kube-api-access-c9m24\") pod \"volume-data-source-validator-78495597f6-6l9p9\" (UID: \"cbd2cf50-f068-4c6f-8f23-a254a4515790\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9"
Jul 12 15:14:04.417361 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.417329    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"]
Jul 12 15:14:04.420219 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.420196    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.423222 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.423198    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-z7p7g\""
Jul 12 15:14:04.424645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.424627    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 12 15:14:04.424645 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.424635    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 12 15:14:04.424821 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.424660    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 12 15:14:04.424821 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.424737    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 12 15:14:04.433375 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.433353    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"]
Jul 12 15:14:04.465144 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.465064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/09036cb5-7b87-4026-bf52-146f44107558-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.465144 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.465097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.465328 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.465156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9m24\" (UniqueName: \"kubernetes.io/projected/cbd2cf50-f068-4c6f-8f23-a254a4515790-kube-api-access-c9m24\") pod \"volume-data-source-validator-78495597f6-6l9p9\" (UID: \"cbd2cf50-f068-4c6f-8f23-a254a4515790\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9"
Jul 12 15:14:04.465328 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.465209    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2glrz\" (UniqueName: \"kubernetes.io/projected/09036cb5-7b87-4026-bf52-146f44107558-kube-api-access-2glrz\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.473059 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.473035    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9m24\" (UniqueName: \"kubernetes.io/projected/cbd2cf50-f068-4c6f-8f23-a254a4515790-kube-api-access-c9m24\") pod \"volume-data-source-validator-78495597f6-6l9p9\" (UID: \"cbd2cf50-f068-4c6f-8f23-a254a4515790\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9"
Jul 12 15:14:04.566070 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.566020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2glrz\" (UniqueName: \"kubernetes.io/projected/09036cb5-7b87-4026-bf52-146f44107558-kube-api-access-2glrz\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.566247 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.566090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/09036cb5-7b87-4026-bf52-146f44107558-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.566247 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.566106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.566247 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:04.566227    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:04.566346 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:04.566290    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls podName:09036cb5-7b87-4026-bf52-146f44107558 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:05.066271896 +0000 UTC m=+263.433872740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6rlp4" (UID: "09036cb5-7b87-4026-bf52-146f44107558") : secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:04.566788 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.566764    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/09036cb5-7b87-4026-bf52-146f44107558-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.576077 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.576054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2glrz\" (UniqueName: \"kubernetes.io/projected/09036cb5-7b87-4026-bf52-146f44107558-kube-api-access-2glrz\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:04.626920 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.626881    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9"
Jul 12 15:14:04.746102 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.746022    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9"]
Jul 12 15:14:04.749163 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:04.749139    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd2cf50_f068_4c6f_8f23_a254a4515790.slice/crio-dccd044a17925aa896326d1129fa952c68d5ad47720fb7d77b18b6f08ffbeedf WatchSource:0}: Error finding container dccd044a17925aa896326d1129fa952c68d5ad47720fb7d77b18b6f08ffbeedf: Status 404 returned error can't find the container with id dccd044a17925aa896326d1129fa952c68d5ad47720fb7d77b18b6f08ffbeedf
Jul 12 15:14:04.886539 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:04.886504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9" event={"ID":"cbd2cf50-f068-4c6f-8f23-a254a4515790","Type":"ContainerStarted","Data":"dccd044a17925aa896326d1129fa952c68d5ad47720fb7d77b18b6f08ffbeedf"}
Jul 12 15:14:05.068823 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:05.068713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:05.068979 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:05.068848    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:05.068979 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:05.068901    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls podName:09036cb5-7b87-4026-bf52-146f44107558 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:06.068886495 +0000 UTC m=+264.436487320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6rlp4" (UID: "09036cb5-7b87-4026-bf52-146f44107558") : secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:06.076212 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:06.076172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:06.076693 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:06.076351    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:06.076693 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:06.076438    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls podName:09036cb5-7b87-4026-bf52-146f44107558 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:08.07641609 +0000 UTC m=+266.444016915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6rlp4" (UID: "09036cb5-7b87-4026-bf52-146f44107558") : secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:06.893917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:06.893882    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9" event={"ID":"cbd2cf50-f068-4c6f-8f23-a254a4515790","Type":"ContainerStarted","Data":"2fe1c25126af70e5b5353e174ad3c7e5f853d3965a7d07e76c4313ddf120c036"}
Jul 12 15:14:08.089731 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:08.089699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:08.090121 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:08.089858    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:08.090121 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:08.089922    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls podName:09036cb5-7b87-4026-bf52-146f44107558 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:12.089908137 +0000 UTC m=+270.457508961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6rlp4" (UID: "09036cb5-7b87-4026-bf52-146f44107558") : secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:09.186000 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.185944    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6l9p9" podStartSLOduration=4.073843504 podStartE2EDuration="5.185927389s" totalImagesPullingTime="1.112083885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:06.907827072 +0000 UTC m=+265.275427912" watchObservedRunningTime="2026-07-12 15:14:09.185927389 +0000 UTC m=+267.553528234"
Jul 12 15:14:09.186386 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.186085    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gk24p"]
Jul 12 15:14:09.189114 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.189095    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.191609 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.191588    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 12 15:14:09.191737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.191718    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 12 15:14:09.191860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.191821    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 12 15:14:09.193037 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.193019    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gr22b\""
Jul 12 15:14:09.193126 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.193022    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 12 15:14:09.198452 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.198361    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gk24p"]
Jul 12 15:14:09.300484 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.300450    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt"]
Jul 12 15:14:09.300639 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.300605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phqns\" (UniqueName: \"kubernetes.io/projected/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-api-access-phqns\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.300682 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.300637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.300782 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.300761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ceb397e-d9a7-470f-bd67-1c70333d4791-data-volume\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.300833 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.300817    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ceb397e-d9a7-470f-bd67-1c70333d4791-crio-socket\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.300885 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.300872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.303450 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.303438    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt"
Jul 12 15:14:09.306026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.305998    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 12 15:14:09.306026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.306009    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 12 15:14:09.306188 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.306051    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dd8fb\""
Jul 12 15:14:09.311737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.311714    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt"]
Jul 12 15:14:09.401567 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.401525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phqns\" (UniqueName: \"kubernetes.io/projected/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-api-access-phqns\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.401763 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.401573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.401763 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.401626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqmlk\" (UniqueName: \"kubernetes.io/projected/c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d-kube-api-access-mqmlk\") pod \"migrator-7f59b54f6c-bj8zt\" (UID: \"c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt"
Jul 12 15:14:09.401763 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.401664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ceb397e-d9a7-470f-bd67-1c70333d4791-data-volume\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.401763 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.401694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ceb397e-d9a7-470f-bd67-1c70333d4791-crio-socket\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.401763 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.401760    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ceb397e-d9a7-470f-bd67-1c70333d4791-crio-socket\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.402026 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:09.401765    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:09.402026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.401782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.402026 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:09.401859    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:09.901839611 +0000 UTC m=+268.269440435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:09.402026 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:09.401883    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 15:14:09.402026 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:09.401935    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:09.901923392 +0000 UTC m=+268.269524221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : configmap "kube-rbac-proxy" not found
Jul 12 15:14:09.402220 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.402132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ceb397e-d9a7-470f-bd67-1c70333d4791-data-volume\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.409421 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.409397    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phqns\" (UniqueName: \"kubernetes.io/projected/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-api-access-phqns\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.503050 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.502970    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqmlk\" (UniqueName: \"kubernetes.io/projected/c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d-kube-api-access-mqmlk\") pod \"migrator-7f59b54f6c-bj8zt\" (UID: \"c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt"
Jul 12 15:14:09.510095 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.510069    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqmlk\" (UniqueName: \"kubernetes.io/projected/c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d-kube-api-access-mqmlk\") pod \"migrator-7f59b54f6c-bj8zt\" (UID: \"c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt"
Jul 12 15:14:09.611514 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.611472    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt"
Jul 12 15:14:09.729223 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.729191    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt"]
Jul 12 15:14:09.732306 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:09.732277    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc33ab7fc_7329_4b24_8c51_7b2fc4f3fd0d.slice/crio-7f86a81f4dd2ae153a66e3ec77137dcd40cd5e266cbe061f48c405fc01ffdfcf WatchSource:0}: Error finding container 7f86a81f4dd2ae153a66e3ec77137dcd40cd5e266cbe061f48c405fc01ffdfcf: Status 404 returned error can't find the container with id 7f86a81f4dd2ae153a66e3ec77137dcd40cd5e266cbe061f48c405fc01ffdfcf
Jul 12 15:14:09.905737 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.905698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.905939 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.905799    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:09.905939 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:09.905816    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:09.905939 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:09.905879    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 15:14:09.905939 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:09.905890    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:10.905867261 +0000 UTC m=+269.273468102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:09.905939 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:09.905934    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:10.905921957 +0000 UTC m=+269.273522781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : configmap "kube-rbac-proxy" not found
Jul 12 15:14:09.906368 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:09.906349    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt" event={"ID":"c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d","Type":"ContainerStarted","Data":"7f86a81f4dd2ae153a66e3ec77137dcd40cd5e266cbe061f48c405fc01ffdfcf"}
Jul 12 15:14:10.915400 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:10.915367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:10.915763 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:10.915422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:10.915763 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:10.915489    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 15:14:10.915763 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:10.915545    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:12.915527151 +0000 UTC m=+271.283127975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : configmap "kube-rbac-proxy" not found
Jul 12 15:14:10.915763 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:10.915603    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:10.915763 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:10.915644    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:12.915633857 +0000 UTC m=+271.283234681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:11.386066 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:11.386043    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lqhsc_3e91ea4b-53f0-4d60-9398-f65b111d3fa0/dns-node-resolver/0.log"
Jul 12 15:14:11.913965 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:11.913934    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt" event={"ID":"c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d","Type":"ContainerStarted","Data":"970dc6d015e70c44dee8b2f893935a531442faaf237667b93d47ca9b6fba96f6"}
Jul 12 15:14:11.914110 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:11.913985    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt" event={"ID":"c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d","Type":"ContainerStarted","Data":"138c47748012734db3bc086bbe6d263721c602a302ef65a54705f44850dc8d48"}
Jul 12 15:14:11.930110 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:11.930063    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-bj8zt" podStartSLOduration=1.7317960669999999 podStartE2EDuration="2.930048164s" totalImagesPullingTime="1.198252097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:11.929234846 +0000 UTC m=+270.296835704" watchObservedRunningTime="2026-07-12 15:14:11.930048164 +0000 UTC m=+270.297649009"
Jul 12 15:14:12.127506 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:12.127457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:12.127643 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:12.127607    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:12.127680 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:12.127668    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls podName:09036cb5-7b87-4026-bf52-146f44107558 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:20.127654296 +0000 UTC m=+278.495255121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6rlp4" (UID: "09036cb5-7b87-4026-bf52-146f44107558") : secret "cluster-monitoring-operator-tls" not found
Jul 12 15:14:12.192869 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:12.192785    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b4mrw_d1190c39-854f-4b29-b0de-b6b6e677834e/node-ca/0.log"
Jul 12 15:14:12.932738 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:12.932703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:12.933172 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:12.932759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:12.933172 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:12.932836    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 15:14:12.933172 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:12.932877    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:12.933172 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:12.932897    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:16.93288089 +0000 UTC m=+275.300481715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : configmap "kube-rbac-proxy" not found
Jul 12 15:14:12.933172 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:12.932920    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:16.932908868 +0000 UTC m=+275.300509693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:12.986602 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:12.986575    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jp65g_c0a29124-e5db-412f-b3fe-5879f9d68c88/serve-healthcheck-canary/0.log"
Jul 12 15:14:16.963771 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:16.963733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:16.964203 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:16.963797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:16.964203 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:16.963869    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 15:14:16.964203 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:16.963938    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:16.964203 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:16.963941    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:24.963920567 +0000 UTC m=+283.331521394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : configmap "kube-rbac-proxy" not found
Jul 12 15:14:16.964203 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:16.964011    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls podName:2ceb397e-d9a7-470f-bd67-1c70333d4791 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:24.963996134 +0000 UTC m=+283.331596964 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gk24p" (UID: "2ceb397e-d9a7-470f-bd67-1c70333d4791") : secret "insights-runtime-extractor-tls" not found
Jul 12 15:14:20.193083 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:20.193037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:20.195392 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:20.195366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/09036cb5-7b87-4026-bf52-146f44107558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6rlp4\" (UID: \"09036cb5-7b87-4026-bf52-146f44107558\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:20.329208 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:20.329154    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"
Jul 12 15:14:20.452252 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:20.452178    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4"]
Jul 12 15:14:20.455131 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:20.455103    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09036cb5_7b87_4026_bf52_146f44107558.slice/crio-de989b9c8b7f0ab6fd84b022ea43aa466c5abdcb1087cb318dfcecc19b1947eb WatchSource:0}: Error finding container de989b9c8b7f0ab6fd84b022ea43aa466c5abdcb1087cb318dfcecc19b1947eb: Status 404 returned error can't find the container with id de989b9c8b7f0ab6fd84b022ea43aa466c5abdcb1087cb318dfcecc19b1947eb
Jul 12 15:14:20.943933 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:20.943900    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4" event={"ID":"09036cb5-7b87-4026-bf52-146f44107558","Type":"ContainerStarted","Data":"de989b9c8b7f0ab6fd84b022ea43aa466c5abdcb1087cb318dfcecc19b1947eb"}
Jul 12 15:14:22.950937 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:22.950898    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4" event={"ID":"09036cb5-7b87-4026-bf52-146f44107558","Type":"ContainerStarted","Data":"aeb6538d0eb224e04ec7baeb76560c861d691ba2a76b99aeafc896c70d3f8e2f"}
Jul 12 15:14:22.966190 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:22.966142    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6rlp4" podStartSLOduration=17.207206337 podStartE2EDuration="18.966126108s" totalImagesPullingTime="1.758919771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:22.965472569 +0000 UTC m=+281.333073415" watchObservedRunningTime="2026-07-12 15:14:22.966126108 +0000 UTC m=+281.333727006"
Jul 12 15:14:25.031221 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.031169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:25.031629 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.031244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:25.031698 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.031680    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ceb397e-d9a7-470f-bd67-1c70333d4791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:25.033499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.033482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ceb397e-d9a7-470f-bd67-1c70333d4791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gk24p\" (UID: \"2ceb397e-d9a7-470f-bd67-1c70333d4791\") " pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:25.097994 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.097951    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gk24p"
Jul 12 15:14:25.221859 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.221827    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gk24p"]
Jul 12 15:14:25.224693 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:25.224657    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ceb397e_d9a7_470f_bd67_1c70333d4791.slice/crio-445fb6740ced26c1b358c3a9610b265357e935111f17870becc5224fdd625153 WatchSource:0}: Error finding container 445fb6740ced26c1b358c3a9610b265357e935111f17870becc5224fdd625153: Status 404 returned error can't find the container with id 445fb6740ced26c1b358c3a9610b265357e935111f17870becc5224fdd625153
Jul 12 15:14:25.643311 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:25.643259    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp" podUID="b2f2c03a-fced-4df6-842b-257fca3c1f18"
Jul 12 15:14:25.643505 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:25.643448    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" podUID="9ee77f29-b901-4674-a7c5-50380d1100e6"
Jul 12 15:14:25.643643 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:25.643621    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-smg4c" podUID="f904686e-b8e3-4353-af52-687df74b253a"
Jul 12 15:14:25.963327 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.963247    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:14:25.963624 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.963601    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-smg4c"
Jul 12 15:14:25.963712 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.963633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gk24p" event={"ID":"2ceb397e-d9a7-470f-bd67-1c70333d4791","Type":"ContainerStarted","Data":"445fb6740ced26c1b358c3a9610b265357e935111f17870becc5224fdd625153"}
Jul 12 15:14:25.963712 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:25.963615    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:14:28.968381 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:28.968343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:14:28.968381 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:28.968389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:14:28.968941 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:28.968441    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:14:28.970965 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:28.970931    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f904686e-b8e3-4353-af52-687df74b253a-metrics-tls\") pod \"dns-default-smg4c\" (UID: \"f904686e-b8e3-4353-af52-687df74b253a\") " pod="openshift-dns/dns-default-smg4c"
Jul 12 15:14:28.971087 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:28.970945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"image-registry-65b85f644b-ss8lm\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") " pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:14:28.971087 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:28.970993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2f2c03a-fced-4df6-842b-257fca3c1f18-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-mjwlp\" (UID: \"b2f2c03a-fced-4df6-842b-257fca3c1f18\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:14:28.974963 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:28.974938    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gk24p" event={"ID":"2ceb397e-d9a7-470f-bd67-1c70333d4791","Type":"ContainerStarted","Data":"b9cf7f1899fccfffbb5f00ac64df9f99ce89ac805abadcab28b8d902a8eaf950"}
Jul 12 15:14:29.267503 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.267466    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-72hfp\""
Jul 12 15:14:29.267674 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.267534    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hsr44\""
Jul 12 15:14:29.267674 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.267466    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-t6qrp\""
Jul 12 15:14:29.274716 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.274688    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-smg4c"
Jul 12 15:14:29.274879 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.274762    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"
Jul 12 15:14:29.274879 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.274783    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:14:29.462762 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.462731    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-mjwlp"]
Jul 12 15:14:29.465380 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:29.465355    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f2c03a_fced_4df6_842b_257fca3c1f18.slice/crio-01924dbb8f294af286ac0dc7a2c65f5919fa78e7fee9d211417023230f186c63 WatchSource:0}: Error finding container 01924dbb8f294af286ac0dc7a2c65f5919fa78e7fee9d211417023230f186c63: Status 404 returned error can't find the container with id 01924dbb8f294af286ac0dc7a2c65f5919fa78e7fee9d211417023230f186c63
Jul 12 15:14:29.684232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.684206    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-smg4c"]
Jul 12 15:14:29.686522 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.686494    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65b85f644b-ss8lm"]
Jul 12 15:14:29.686680 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:29.686652    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf904686e_b8e3_4353_af52_687df74b253a.slice/crio-e4277f791580e53085894bad6ad8361afde786729c0f98a7ffd57110f3132875 WatchSource:0}: Error finding container e4277f791580e53085894bad6ad8361afde786729c0f98a7ffd57110f3132875: Status 404 returned error can't find the container with id e4277f791580e53085894bad6ad8361afde786729c0f98a7ffd57110f3132875
Jul 12 15:14:29.689354 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:29.689336    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee77f29_b901_4674_a7c5_50380d1100e6.slice/crio-f2c52452e1affe6ec30338d12973938206c6c1f62ff6e284c59b3af6edab3103 WatchSource:0}: Error finding container f2c52452e1affe6ec30338d12973938206c6c1f62ff6e284c59b3af6edab3103: Status 404 returned error can't find the container with id f2c52452e1affe6ec30338d12973938206c6c1f62ff6e284c59b3af6edab3103
Jul 12 15:14:29.980007 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.979911    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp" event={"ID":"b2f2c03a-fced-4df6-842b-257fca3c1f18","Type":"ContainerStarted","Data":"01924dbb8f294af286ac0dc7a2c65f5919fa78e7fee9d211417023230f186c63"}
Jul 12 15:14:29.981970 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.981868    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" event={"ID":"9ee77f29-b901-4674-a7c5-50380d1100e6","Type":"ContainerStarted","Data":"040407103acb9bd5724ddbfe7fe97f8189d10b634154f09b4d60ae613a37c4ed"}
Jul 12 15:14:29.981970 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.981918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" event={"ID":"9ee77f29-b901-4674-a7c5-50380d1100e6","Type":"ContainerStarted","Data":"f2c52452e1affe6ec30338d12973938206c6c1f62ff6e284c59b3af6edab3103"}
Jul 12 15:14:29.982168 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.981989    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:14:29.983514 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.983488    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smg4c" event={"ID":"f904686e-b8e3-4353-af52-687df74b253a","Type":"ContainerStarted","Data":"e4277f791580e53085894bad6ad8361afde786729c0f98a7ffd57110f3132875"}
Jul 12 15:14:29.985620 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:29.985591    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gk24p" event={"ID":"2ceb397e-d9a7-470f-bd67-1c70333d4791","Type":"ContainerStarted","Data":"063f177ea8c506508b4059162139c35b79ec6d7e25cc973f0b7f9b9f33b80301"}
Jul 12 15:14:30.004233 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:30.004170    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" podStartSLOduration=288.004150159 podStartE2EDuration="4m48.004150159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:30.002946948 +0000 UTC m=+288.370547796" watchObservedRunningTime="2026-07-12 15:14:30.004150159 +0000 UTC m=+288.371751007"
Jul 12 15:14:30.991275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:30.991219    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp" event={"ID":"b2f2c03a-fced-4df6-842b-257fca3c1f18","Type":"ContainerStarted","Data":"a277c33de4064ca8862070f5c3e44492d2255c73a99ed4b10691482ce722616c"}
Jul 12 15:14:31.008178 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:31.008125    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-mjwlp" podStartSLOduration=276.588631379 podStartE2EDuration="4m38.008108518s" totalImagesPullingTime="1.419477139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:31.007249519 +0000 UTC m=+289.374850366" watchObservedRunningTime="2026-07-12 15:14:31.008108518 +0000 UTC m=+289.375709364"
Jul 12 15:14:31.962327 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:31.962286    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"]
Jul 12 15:14:31.978930 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:31.978899    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"]
Jul 12 15:14:31.979116 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:31.979028    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"
Jul 12 15:14:31.981501 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:31.981475    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-twmw2\""
Jul 12 15:14:31.982818 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:31.982779    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 12 15:14:32.098759 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:32.098716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/62e21818-34d3-4c2e-beda-994c2fe01a47-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-qt6mv\" (UID: \"62e21818-34d3-4c2e-beda-994c2fe01a47\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"
Jul 12 15:14:32.199256 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:32.199224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/62e21818-34d3-4c2e-beda-994c2fe01a47-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-qt6mv\" (UID: \"62e21818-34d3-4c2e-beda-994c2fe01a47\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"
Jul 12 15:14:32.202151 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:32.202118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/62e21818-34d3-4c2e-beda-994c2fe01a47-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-qt6mv\" (UID: \"62e21818-34d3-4c2e-beda-994c2fe01a47\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"
Jul 12 15:14:32.291022 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:32.290931    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"
Jul 12 15:14:32.715141 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:32.715017    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"]
Jul 12 15:14:32.719893 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:32.719861    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62e21818_34d3_4c2e_beda_994c2fe01a47.slice/crio-88073e5f841016643f4464397a78276b97a2bfd3c78946972628692e2222ffcd WatchSource:0}: Error finding container 88073e5f841016643f4464397a78276b97a2bfd3c78946972628692e2222ffcd: Status 404 returned error can't find the container with id 88073e5f841016643f4464397a78276b97a2bfd3c78946972628692e2222ffcd
Jul 12 15:14:33.000001 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:32.999954    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smg4c" event={"ID":"f904686e-b8e3-4353-af52-687df74b253a","Type":"ContainerStarted","Data":"1d4347dce1fd368ed92882eca3c207aa768628a67ad26e10a43c821905d96ffa"}
Jul 12 15:14:33.000191 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:33.000097    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-smg4c"
Jul 12 15:14:33.000191 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:33.000151    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smg4c" event={"ID":"f904686e-b8e3-4353-af52-687df74b253a","Type":"ContainerStarted","Data":"f4ac797ffc94f4bc3741d2c23a7d2e07fc01cb72dd4cdce9777856a0fee5e8ac"}
Jul 12 15:14:33.001875 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:33.001853    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gk24p" event={"ID":"2ceb397e-d9a7-470f-bd67-1c70333d4791","Type":"ContainerStarted","Data":"9c2f27ebca5e85b079cf5ec1c41af985461effb4fee6df0a5532e45cbacae23e"}
Jul 12 15:14:33.003019 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:33.003001    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv" event={"ID":"62e21818-34d3-4c2e-beda-994c2fe01a47","Type":"ContainerStarted","Data":"88073e5f841016643f4464397a78276b97a2bfd3c78946972628692e2222ffcd"}
Jul 12 15:14:33.016643 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:33.016605    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-smg4c" podStartSLOduration=252.121619123 podStartE2EDuration="4m15.016592604s" totalImagesPullingTime="2.894973481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:33.014832779 +0000 UTC m=+291.382433615" watchObservedRunningTime="2026-07-12 15:14:33.016592604 +0000 UTC m=+291.384193428"
Jul 12 15:14:33.030904 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:33.030844    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gk24p" podStartSLOduration=16.808017343 podStartE2EDuration="24.030824987s" totalImagesPullingTime="7.222807644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:14:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:33.030033044 +0000 UTC m=+291.397633883" watchObservedRunningTime="2026-07-12 15:14:33.030824987 +0000 UTC m=+291.398425834"
Jul 12 15:14:34.007858 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:34.007792    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv" event={"ID":"62e21818-34d3-4c2e-beda-994c2fe01a47","Type":"ContainerStarted","Data":"73834eff9080b7572a55fe94047bfdd0eab10de99fb7d931c476f77905335365"}
Jul 12 15:14:34.008468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:34.008246    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"
Jul 12 15:14:34.012427 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:34.012404    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv"
Jul 12 15:14:34.022499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:34.022460    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qt6mv" podStartSLOduration=1.877304091 podStartE2EDuration="3.022447609s" totalImagesPullingTime="1.145143518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:34.021891405 +0000 UTC m=+292.389492251" watchObservedRunningTime="2026-07-12 15:14:34.022447609 +0000 UTC m=+292.390048454"
Jul 12 15:14:38.439203 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.439167    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c4b9b766d-4h6dr"]
Jul 12 15:14:38.444885 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.444859    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.447265 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.447235    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 12 15:14:38.447483 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.447468    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wwtcp\""
Jul 12 15:14:38.447569 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.447551    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 12 15:14:38.448562 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.448545    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 12 15:14:38.448676 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.448582    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 12 15:14:38.448676 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.448584    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 12 15:14:38.448676 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.448621    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 12 15:14:38.448831 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.448709    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 12 15:14:38.452921 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.452894    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4b9b766d-4h6dr"]
Jul 12 15:14:38.543118 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.543075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-console-config\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.543118 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.543116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-oauth-serving-cert\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.543324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.543143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-oauth-config\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.543324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.543189    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-serving-cert\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.543324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.543216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-service-ca\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.543324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.543234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5r5t\" (UniqueName: \"kubernetes.io/projected/d1758264-b8b2-42ef-9c51-35929219892a-kube-api-access-x5r5t\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.643764 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.643727    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-console-config\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.643764 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.643766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-oauth-serving-cert\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.644055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.643800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-oauth-config\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.644055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.643851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-serving-cert\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.644055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.643877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-service-ca\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.644055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.643900    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5r5t\" (UniqueName: \"kubernetes.io/projected/d1758264-b8b2-42ef-9c51-35929219892a-kube-api-access-x5r5t\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.644551 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.644527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-console-config\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.644651 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.644610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-oauth-serving-cert\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.644651 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.644630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-service-ca\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.646273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.646254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-oauth-config\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.646347 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.646319    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-serving-cert\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.656482 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.656457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5r5t\" (UniqueName: \"kubernetes.io/projected/d1758264-b8b2-42ef-9c51-35929219892a-kube-api-access-x5r5t\") pod \"console-5c4b9b766d-4h6dr\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") " pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.754982 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.754897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:38.889927 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:38.889769    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4b9b766d-4h6dr"]
Jul 12 15:14:38.892462 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:38.892437    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1758264_b8b2_42ef_9c51_35929219892a.slice/crio-a9e4519b92a339c85a7fc1cbfb188401cdf0b12e8566d4c4a4bf72a338af6596 WatchSource:0}: Error finding container a9e4519b92a339c85a7fc1cbfb188401cdf0b12e8566d4c4a4bf72a338af6596: Status 404 returned error can't find the container with id a9e4519b92a339c85a7fc1cbfb188401cdf0b12e8566d4c4a4bf72a338af6596
Jul 12 15:14:39.026720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.026636    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4b9b766d-4h6dr" event={"ID":"d1758264-b8b2-42ef-9c51-35929219892a","Type":"ContainerStarted","Data":"a9e4519b92a339c85a7fc1cbfb188401cdf0b12e8566d4c4a4bf72a338af6596"}
Jul 12 15:14:39.079730 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.079697    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8npnz"]
Jul 12 15:14:39.086508 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.086478    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.088908 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.088883    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cxgsr\""
Jul 12 15:14:39.089020 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.088933    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 12 15:14:39.089089 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.089073    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 12 15:14:39.089132 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.089107    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 12 15:14:39.089197 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.089182    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 12 15:14:39.147523 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-sys\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.147670 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-accelerators-collector-config\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.147670 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-metrics-client-ca\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.147670 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.147670 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-wtmp\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.147670 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147646    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-textfile\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.147670 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-tls\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.147902 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147690    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnc6f\" (UniqueName: \"kubernetes.io/projected/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-kube-api-access-rnc6f\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.147902 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.147705    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-root\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.150065 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:39.150026    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6vsjp" podUID="dcc058fa-537e-49c8-a4b9-79fe8986f083"
Jul 12 15:14:39.249014 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.248981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-sys\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249014 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249018    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-accelerators-collector-config\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249049    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-metrics-client-ca\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249092    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-wtmp\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-textfile\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-sys\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-tls\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249232 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:39.249203    2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 12 15:14:39.249232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249228    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnc6f\" (UniqueName: \"kubernetes.io/projected/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-kube-api-access-rnc6f\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249602 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:14:39.249252    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-tls podName:2801364e-bb09-487c-9f1b-2bbb4ae9c9a0 nodeName:}" failed. No retries permitted until 2026-07-12 15:14:39.749238786 +0000 UTC m=+298.116839611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-tls") pod "node-exporter-8npnz" (UID: "2801364e-bb09-487c-9f1b-2bbb4ae9c9a0") : secret "node-exporter-tls" not found
Jul 12 15:14:39.249602 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249256    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-wtmp\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249602 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-root\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249602 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-root\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249602 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249531    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-textfile\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249756 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-accelerators-collector-config\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.249756 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.249737    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-metrics-client-ca\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.251416 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.251399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.262422 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.262399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnc6f\" (UniqueName: \"kubernetes.io/projected/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-kube-api-access-rnc6f\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.753501 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.753455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-tls\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.756085 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.756062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2801364e-bb09-487c-9f1b-2bbb4ae9c9a0-node-exporter-tls\") pod \"node-exporter-8npnz\" (UID: \"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0\") " pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:39.995335 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:39.995299    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8npnz"
Jul 12 15:14:40.011061 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:40.011026    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2801364e_bb09_487c_9f1b_2bbb4ae9c9a0.slice/crio-12d661517001320363352b2d33ef6b440665df3c7d8037f3ea9096f6c004f9f7 WatchSource:0}: Error finding container 12d661517001320363352b2d33ef6b440665df3c7d8037f3ea9096f6c004f9f7: Status 404 returned error can't find the container with id 12d661517001320363352b2d33ef6b440665df3c7d8037f3ea9096f6c004f9f7
Jul 12 15:14:40.035795 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.035745    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8npnz" event={"ID":"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0","Type":"ContainerStarted","Data":"12d661517001320363352b2d33ef6b440665df3c7d8037f3ea9096f6c004f9f7"}
Jul 12 15:14:40.114384 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.114350    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:14:40.119842 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.119798    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.122432 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.122353    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 12 15:14:40.122432 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.122357    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 12 15:14:40.122432 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.122399    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 12 15:14:40.122653 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.122435    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vvgds\""
Jul 12 15:14:40.122721 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.122694    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 12 15:14:40.122771 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.122736    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 12 15:14:40.124118 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.123932    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 12 15:14:40.124118 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.123944    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 12 15:14:40.124118 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.123945    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 12 15:14:40.124118 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.124057    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 12 15:14:40.131058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.131034    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:14:40.157251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157415 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157415 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-out\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157415 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-web-config\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157556 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157414    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157556 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157510    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157556 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157658 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157573    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjkqx\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-kube-api-access-gjkqx\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157658 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157663    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.157922 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.157789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjkqx\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-kube-api-access-gjkqx\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.259428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.260108 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-out\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.260108 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-web-config\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.260108 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.260108 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.259937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.261364 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.260895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.261364 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.261187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.263851 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.263772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.265273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.264982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.265273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.265050    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.265273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.265141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.265273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.265220    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-out\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.265533 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.265478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.266281 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.266256    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.266791 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.266766    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.267105 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.267085    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-web-config\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.268503 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.268483    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjkqx\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-kube-api-access-gjkqx\") pod \"alertmanager-main-0\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.432038 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.431993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:14:40.593504 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:40.593455    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:14:40.597448 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:40.597418    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9eee1b1_cfe9_4d25_9be8_bc2805e359f1.slice/crio-5c1892a854bdaa9f5cc5feef8ea891dc53599bb4911caebc301f915044a9f373 WatchSource:0}: Error finding container 5c1892a854bdaa9f5cc5feef8ea891dc53599bb4911caebc301f915044a9f373: Status 404 returned error can't find the container with id 5c1892a854bdaa9f5cc5feef8ea891dc53599bb4911caebc301f915044a9f373
Jul 12 15:14:41.031666 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.031628    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"]
Jul 12 15:14:41.035792 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.035762    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.038266 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.038152    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 12 15:14:41.038419 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.038263    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1la879r67jftk\""
Jul 12 15:14:41.038419 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.038269    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 12 15:14:41.038568 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.038548    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 12 15:14:41.038646 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.038620    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bbh49\""
Jul 12 15:14:41.038719 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.038674    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 12 15:14:41.038970 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.038949    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 12 15:14:41.042056 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.042030    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerStarted","Data":"5c1892a854bdaa9f5cc5feef8ea891dc53599bb4911caebc301f915044a9f373"}
Jul 12 15:14:41.045633 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.045613    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"]
Jul 12 15:14:41.068782 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.068749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv7xs\" (UniqueName: \"kubernetes.io/projected/0d38f416-4fb6-4de7-8e67-b005f9be5f66-kube-api-access-fv7xs\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.068988 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.068793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.068988 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.068880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.068988 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.068913    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d38f416-4fb6-4de7-8e67-b005f9be5f66-metrics-client-ca\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.068988 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.068949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-grpc-tls\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.069206 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.069023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-tls\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.069206 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.069061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.069206 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.069095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.169991 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.169950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv7xs\" (UniqueName: \"kubernetes.io/projected/0d38f416-4fb6-4de7-8e67-b005f9be5f66-kube-api-access-fv7xs\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.170189 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.170006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.170189 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.170047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.170189 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.170069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d38f416-4fb6-4de7-8e67-b005f9be5f66-metrics-client-ca\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.170189 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.170103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-grpc-tls\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.170189 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.170140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-tls\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.170189 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.170164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.170492 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.170191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.171035 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.170985    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d38f416-4fb6-4de7-8e67-b005f9be5f66-metrics-client-ca\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.173407 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.173362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-tls\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.174002 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.173952    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.174117 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.174020    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.174215 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.174192    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.174577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.174556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-grpc-tls\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.174652 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.174580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0d38f416-4fb6-4de7-8e67-b005f9be5f66-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.177975 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.177952    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv7xs\" (UniqueName: \"kubernetes.io/projected/0d38f416-4fb6-4de7-8e67-b005f9be5f66-kube-api-access-fv7xs\") pod \"thanos-querier-5bf7f4bb45-5hdf2\" (UID: \"0d38f416-4fb6-4de7-8e67-b005f9be5f66\") " pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:41.347756 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:41.347706    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:42.090459 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:42.090421    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 12 15:14:42.118718 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:42.118688    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"]
Jul 12 15:14:42.121942 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:42.121900    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d38f416_4fb6_4de7_8e67_b005f9be5f66.slice/crio-22dab8d7bb2c5030d916cdeb91567079d8701ccd7bb2a82738a4f662486b5331 WatchSource:0}: Error finding container 22dab8d7bb2c5030d916cdeb91567079d8701ccd7bb2a82738a4f662486b5331: Status 404 returned error can't find the container with id 22dab8d7bb2c5030d916cdeb91567079d8701ccd7bb2a82738a4f662486b5331
Jul 12 15:14:43.010826 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.010773    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-smg4c"
Jul 12 15:14:43.053175 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.053131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2" event={"ID":"0d38f416-4fb6-4de7-8e67-b005f9be5f66","Type":"ContainerStarted","Data":"22dab8d7bb2c5030d916cdeb91567079d8701ccd7bb2a82738a4f662486b5331"}
Jul 12 15:14:43.054991 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.054959    2578 generic.go:356] "Generic (PLEG): container finished" podID="2801364e-bb09-487c-9f1b-2bbb4ae9c9a0" containerID="fd8ccae6da308f665b0fb61ab7b96164488beb9972372de7e75f9490305b575f" exitCode=0
Jul 12 15:14:43.055128 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.055046    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8npnz" event={"ID":"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0","Type":"ContainerDied","Data":"fd8ccae6da308f665b0fb61ab7b96164488beb9972372de7e75f9490305b575f"}
Jul 12 15:14:43.056871 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.056847    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerID="aa67b9c2939d450addec9a98c160c3e3ac13923199eb0678e637d118f7675731" exitCode=0
Jul 12 15:14:43.056981 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.056918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerDied","Data":"aa67b9c2939d450addec9a98c160c3e3ac13923199eb0678e637d118f7675731"}
Jul 12 15:14:43.060665 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.060523    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4b9b766d-4h6dr" event={"ID":"d1758264-b8b2-42ef-9c51-35929219892a","Type":"ContainerStarted","Data":"f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79"}
Jul 12 15:14:43.120668 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.120614    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c4b9b766d-4h6dr" podStartSLOduration=2.038178691 podStartE2EDuration="5.120593896s" totalImagesPullingTime="3.082415205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:43.120019174 +0000 UTC m=+301.487620011" watchObservedRunningTime="2026-07-12 15:14:43.120593896 +0000 UTC m=+301.488194742"
Jul 12 15:14:43.831612 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.831577    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"]
Jul 12 15:14:43.834952 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.834931    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:43.837352 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.837326    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 12 15:14:43.837352 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.837350    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tppnv\""
Jul 12 15:14:43.844297 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.844243    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"]
Jul 12 15:14:43.896122 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.896066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac3bb11b-8b80-4c5f-98af-fc1700bab948-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-x7qh9\" (UID: \"ac3bb11b-8b80-4c5f-98af-fc1700bab948\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:43.896300 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.896131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftd6l\" (UniqueName: \"kubernetes.io/projected/ac3bb11b-8b80-4c5f-98af-fc1700bab948-kube-api-access-ftd6l\") pod \"monitoring-plugin-5f68df7f58-x7qh9\" (UID: \"ac3bb11b-8b80-4c5f-98af-fc1700bab948\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:43.996665 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.996626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftd6l\" (UniqueName: \"kubernetes.io/projected/ac3bb11b-8b80-4c5f-98af-fc1700bab948-kube-api-access-ftd6l\") pod \"monitoring-plugin-5f68df7f58-x7qh9\" (UID: \"ac3bb11b-8b80-4c5f-98af-fc1700bab948\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:43.996894 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.996751    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac3bb11b-8b80-4c5f-98af-fc1700bab948-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-x7qh9\" (UID: \"ac3bb11b-8b80-4c5f-98af-fc1700bab948\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:43.999741 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:43.999717    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac3bb11b-8b80-4c5f-98af-fc1700bab948-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-x7qh9\" (UID: \"ac3bb11b-8b80-4c5f-98af-fc1700bab948\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:44.004678 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:44.004652    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftd6l\" (UniqueName: \"kubernetes.io/projected/ac3bb11b-8b80-4c5f-98af-fc1700bab948-kube-api-access-ftd6l\") pod \"monitoring-plugin-5f68df7f58-x7qh9\" (UID: \"ac3bb11b-8b80-4c5f-98af-fc1700bab948\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:44.068871 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:44.068830    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8npnz" event={"ID":"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0","Type":"ContainerStarted","Data":"cbf6e58b48a227ad3042ceb7ea5d405b5183547cd8330728be52beb364e62d7b"}
Jul 12 15:14:44.069051 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:44.068886    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8npnz" event={"ID":"2801364e-bb09-487c-9f1b-2bbb4ae9c9a0","Type":"ContainerStarted","Data":"fcf1c8052473946acc25d137fd749cc696767192721b541e1f86017d5e561c7d"}
Jul 12 15:14:44.087285 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:44.087177    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8npnz" podStartSLOduration=3.046075311 podStartE2EDuration="5.087153933s" totalImagesPullingTime="1.957235938s" totalInitContainerRuntime="83.842684ms" isStatefulPod=true podCreationTimestamp="2026-07-12 15:14:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:44.085062233 +0000 UTC m=+302.452663092" watchObservedRunningTime="2026-07-12 15:14:44.087153933 +0000 UTC m=+302.454754781"
Jul 12 15:14:44.146008 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:44.145968    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:44.283412 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:44.283378    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"]
Jul 12 15:14:44.526531 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:44.526459    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac3bb11b_8b80_4c5f_98af_fc1700bab948.slice/crio-f6b1e091d7d880a6654f5daece3496b2729828b10c17771fe5749deabdb813a4 WatchSource:0}: Error finding container f6b1e091d7d880a6654f5daece3496b2729828b10c17771fe5749deabdb813a4: Status 404 returned error can't find the container with id f6b1e091d7d880a6654f5daece3496b2729828b10c17771fe5749deabdb813a4
Jul 12 15:14:44.528533 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:44.528513    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 15:14:45.075755 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.075661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerStarted","Data":"e4e5009c6292653bbec7bd9b43dc15cbc1b8bbef9201e0f00706f87fb6141d34"}
Jul 12 15:14:45.075755 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.075737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerStarted","Data":"9d03d77f47746eed2e00af10ab16891870b1ff3502b8d1731ce808968cc2dca5"}
Jul 12 15:14:45.075990 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.075777    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerStarted","Data":"e191ecae29f3c9f3a5abd92cbb95ccc8fdfda4d6ef8afbebc6ab7b96e9342b50"}
Jul 12 15:14:45.075990 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.075842    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerStarted","Data":"f08136ea1cdf509b19b493baf1958fcfc004d337be4d359cbc8bad6ead733b80"}
Jul 12 15:14:45.075990 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.075878    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerStarted","Data":"21e185f360808c0b60172aa8c6e34b2326b0ac3d1c1743aa485546291e49691d"}
Jul 12 15:14:45.077545 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.077518    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2" event={"ID":"0d38f416-4fb6-4de7-8e67-b005f9be5f66","Type":"ContainerStarted","Data":"6951489d12e4175ac48c1c430a51ef4504c1793b3bd544fe467b5a7fa0866091"}
Jul 12 15:14:45.077642 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.077582    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2" event={"ID":"0d38f416-4fb6-4de7-8e67-b005f9be5f66","Type":"ContainerStarted","Data":"6f0a49394725c9a913257d292543d71a26050dc325ec2bad97286c537cdb79d8"}
Jul 12 15:14:45.077642 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.077621    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2" event={"ID":"0d38f416-4fb6-4de7-8e67-b005f9be5f66","Type":"ContainerStarted","Data":"0b36c3c4377c82f56dbacd63f6fb055af0c10b3f67da5d32c20434c6605db980"}
Jul 12 15:14:45.078497 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.078473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9" event={"ID":"ac3bb11b-8b80-4c5f-98af-fc1700bab948","Type":"ContainerStarted","Data":"f6b1e091d7d880a6654f5daece3496b2729828b10c17771fe5749deabdb813a4"}
Jul 12 15:14:45.190131 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.190089    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:14:45.194693 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.194665    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.197156 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.197109    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 12 15:14:45.197288 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.197168    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qr24q\""
Jul 12 15:14:45.197472 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.197299    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 12 15:14:45.197584 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.197558    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 12 15:14:45.197687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.197643    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 12 15:14:45.198358 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.198335    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 12 15:14:45.198444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.198343    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 12 15:14:45.198444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.198392    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 12 15:14:45.198592 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.198537    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 12 15:14:45.199298 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.199115    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 12 15:14:45.199298 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.199141    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 12 15:14:45.199298 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.199162    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 12 15:14:45.199298 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.199189    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 12 15:14:45.199298 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.199118    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 12 15:14:45.199298 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.199115    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1pvt3j211dbgi\""
Jul 12 15:14:45.199955 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.199738    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 12 15:14:45.200641 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.200622    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 12 15:14:45.208447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.208423    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:14:45.211016 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.210989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211123 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211123 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211074    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211139    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211209    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211259    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211282    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-web-config\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config-out\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211434    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211480    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211971 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211570    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.211971 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.211592    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhr6c\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-kube-api-access-vhr6c\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.312758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.312723    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.312758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.312767    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-web-config\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313002 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.312800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313002 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.312858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313002 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.312888    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config-out\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313002 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.312916    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313002 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.312939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313021    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313209    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhr6c\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-kube-api-access-vhr6c\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313375    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.313900 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.313789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.314637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.314006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.314637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.314166    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.314637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.314378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.315267 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.315240    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.315800 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.315769    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.316687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.316226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.316687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.316633    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-web-config\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.317234 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.317210    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.318151 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.318128    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config-out\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.318539 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.318516    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.319044 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.318995    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.319691 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.319636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.320242 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.320178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.321410 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.321167    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.321410 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.321352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.321680 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.321578    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.321680 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.321627    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.322220 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.322198    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.322565 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.322537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhr6c\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-kube-api-access-vhr6c\") pod \"prometheus-k8s-0\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:45.507319 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:45.507270    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:46.090435 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:46.088845    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2" event={"ID":"0d38f416-4fb6-4de7-8e67-b005f9be5f66","Type":"ContainerStarted","Data":"3222f63e1720abaaf926139a7a683bab45668511d77b7a88136d8b62871e044a"}
Jul 12 15:14:46.093799 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:46.093651    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:46.095799 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:46.095692    2578 patch_prober.go:28] interesting pod/monitoring-plugin-5f68df7f58-x7qh9 container/monitoring-plugin namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.132.0.19:9443/health\": dial tcp 10.132.0.19:9443: connect: connection refused" start-of-body=
Jul 12 15:14:46.095799 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:46.095742    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9" podUID="ac3bb11b-8b80-4c5f-98af-fc1700bab948" containerName="monitoring-plugin" probeResult="failure" output="Get \"https://10.132.0.19:9443/health\": dial tcp 10.132.0.19:9443: connect: connection refused"
Jul 12 15:14:46.109666 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:46.109612    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9" podStartSLOduration=1.6421826259999999 podStartE2EDuration="3.109591344s" totalImagesPullingTime="1.467408718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:46.108035473 +0000 UTC m=+304.475636319" watchObservedRunningTime="2026-07-12 15:14:46.109591344 +0000 UTC m=+304.477192192"
Jul 12 15:14:46.134468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:46.134426    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.392404019 podStartE2EDuration="6.13440941s" totalImagesPullingTime="4.726945858s" totalInitContainerRuntime="15.059533ms" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:46.131158397 +0000 UTC m=+304.498759242" watchObservedRunningTime="2026-07-12 15:14:46.13440941 +0000 UTC m=+304.502010256"
Jul 12 15:14:46.147722 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:46.147699    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:14:46.152347 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:14:46.152316    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc4e00ac_07b0_4f01_a3ad_d36c7281e511.slice/crio-302a5d74fe194a2968b9b1873588c9ab691e72fc9a9177ea825f62a094f5157c WatchSource:0}: Error finding container 302a5d74fe194a2968b9b1873588c9ab691e72fc9a9177ea825f62a094f5157c: Status 404 returned error can't find the container with id 302a5d74fe194a2968b9b1873588c9ab691e72fc9a9177ea825f62a094f5157c
Jul 12 15:14:47.102404 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.102370    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231" exitCode=0
Jul 12 15:14:47.102841 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.102453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerDied","Data":"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"}
Jul 12 15:14:47.102841 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.102564    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerStarted","Data":"302a5d74fe194a2968b9b1873588c9ab691e72fc9a9177ea825f62a094f5157c"}
Jul 12 15:14:47.105692 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.105669    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2" event={"ID":"0d38f416-4fb6-4de7-8e67-b005f9be5f66","Type":"ContainerStarted","Data":"9875ca456c0ede0cb573c2f5cb279d459d279779a95da2c699c848ab935ee7ed"}
Jul 12 15:14:47.105787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.105731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2" event={"ID":"0d38f416-4fb6-4de7-8e67-b005f9be5f66","Type":"ContainerStarted","Data":"df227dabd380e8751725949e114f79c8a48d992dca4d1ee89facf7f5ac321b4c"}
Jul 12 15:14:47.105787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.105778    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:47.108953 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.108930    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerStarted","Data":"bad83f3aaa5554a95e54f0bc158c13c75b212d8493e756f9944681de84dbfd52"}
Jul 12 15:14:47.110340 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.110317    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9" event={"ID":"ac3bb11b-8b80-4c5f-98af-fc1700bab948","Type":"ContainerStarted","Data":"0f713ed77fc40672974512f28c93d1176a64d0068da8cb44c03378fc248a6fb6"}
Jul 12 15:14:47.114558 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.114533    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-x7qh9"
Jul 12 15:14:47.154854 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:47.154786    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2" podStartSLOduration=2.509536841 podStartE2EDuration="6.15477011s" totalImagesPullingTime="3.645233269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:47.154278941 +0000 UTC m=+305.521879788" watchObservedRunningTime="2026-07-12 15:14:47.15477011 +0000 UTC m=+305.522370955"
Jul 12 15:14:48.755637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:48.755595    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:48.756125 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:48.755660    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:48.761342 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:48.761303    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:49.125687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:49.125660    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:14:49.280450 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:49.280409    2578 patch_prober.go:28] interesting pod/image-registry-65b85f644b-ss8lm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 12 15:14:49.280639 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:49.280473    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" podUID="9ee77f29-b901-4674-a7c5-50380d1100e6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 12 15:14:50.126657 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:50.126619    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerStarted","Data":"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"}
Jul 12 15:14:50.149718 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:50.149692    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:14:50.996404 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:50.996368    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:14:51.134493 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:51.134455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerStarted","Data":"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"}
Jul 12 15:14:51.134860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:51.134525    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerStarted","Data":"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"}
Jul 12 15:14:51.134860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:51.134553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerStarted","Data":"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"}
Jul 12 15:14:51.134860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:51.134576    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerStarted","Data":"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"}
Jul 12 15:14:51.134860 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:51.134603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerStarted","Data":"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"}
Jul 12 15:14:51.161231 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:51.161173    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.224671575 podStartE2EDuration="6.161156724s" totalImagesPullingTime="2.918003106s" totalInitContainerRuntime="18.482043ms" isStatefulPod=false podCreationTimestamp="2026-07-12 15:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:14:51.159100241 +0000 UTC m=+309.526701087" watchObservedRunningTime="2026-07-12 15:14:51.161156724 +0000 UTC m=+309.528757619"
Jul 12 15:14:53.121858 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:53.121827    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5bf7f4bb45-5hdf2"
Jul 12 15:14:53.811581 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:53.811544    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65b85f644b-ss8lm"]
Jul 12 15:14:55.507878 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:55.507829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:14:56.633534 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:14:56.633501    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c4b9b766d-4h6dr"]
Jul 12 15:15:18.831476 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:18.831405    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" podUID="9ee77f29-b901-4674-a7c5-50380d1100e6" containerName="registry" containerID="cri-o://040407103acb9bd5724ddbfe7fe97f8189d10b634154f09b4d60ae613a37c4ed" gracePeriod=30
Jul 12 15:15:19.246660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.246574    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ee77f29-b901-4674-a7c5-50380d1100e6" containerID="040407103acb9bd5724ddbfe7fe97f8189d10b634154f09b4d60ae613a37c4ed" exitCode=0
Jul 12 15:15:19.246660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.246646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" event={"ID":"9ee77f29-b901-4674-a7c5-50380d1100e6","Type":"ContainerDied","Data":"040407103acb9bd5724ddbfe7fe97f8189d10b634154f09b4d60ae613a37c4ed"}
Jul 12 15:15:19.618232 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.618197    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:15:19.638925 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.638893    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-bound-sa-token\") pod \"9ee77f29-b901-4674-a7c5-50380d1100e6\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") "
Jul 12 15:15:19.639084 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.638951    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-installation-pull-secrets\") pod \"9ee77f29-b901-4674-a7c5-50380d1100e6\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") "
Jul 12 15:15:19.639084 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.638983    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") pod \"9ee77f29-b901-4674-a7c5-50380d1100e6\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") "
Jul 12 15:15:19.639084 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.639015    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-image-registry-private-configuration\") pod \"9ee77f29-b901-4674-a7c5-50380d1100e6\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") "
Jul 12 15:15:19.639084 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.639061    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bl77\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-kube-api-access-8bl77\") pod \"9ee77f29-b901-4674-a7c5-50380d1100e6\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") "
Jul 12 15:15:19.639241 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.639091    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ee77f29-b901-4674-a7c5-50380d1100e6-ca-trust-extracted\") pod \"9ee77f29-b901-4674-a7c5-50380d1100e6\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") "
Jul 12 15:15:19.639241 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.639121    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-certificates\") pod \"9ee77f29-b901-4674-a7c5-50380d1100e6\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") "
Jul 12 15:15:19.639241 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.639161    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-trusted-ca\") pod \"9ee77f29-b901-4674-a7c5-50380d1100e6\" (UID: \"9ee77f29-b901-4674-a7c5-50380d1100e6\") "
Jul 12 15:15:19.640078 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.640049    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9ee77f29-b901-4674-a7c5-50380d1100e6" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:15:19.640920 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.640764    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9ee77f29-b901-4674-a7c5-50380d1100e6" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:15:19.642474 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.642426    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-kube-api-access-8bl77" (OuterVolumeSpecName: "kube-api-access-8bl77") pod "9ee77f29-b901-4674-a7c5-50380d1100e6" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6"). InnerVolumeSpecName "kube-api-access-8bl77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:15:19.642610 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.642509    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9ee77f29-b901-4674-a7c5-50380d1100e6" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:15:19.642610 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.642519    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9ee77f29-b901-4674-a7c5-50380d1100e6" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:15:19.643063 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.643036    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9ee77f29-b901-4674-a7c5-50380d1100e6" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:15:19.643167 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.643138    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9ee77f29-b901-4674-a7c5-50380d1100e6" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:15:19.648874 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.648847    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee77f29-b901-4674-a7c5-50380d1100e6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9ee77f29-b901-4674-a7c5-50380d1100e6" (UID: "9ee77f29-b901-4674-a7c5-50380d1100e6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:15:19.740034 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.740001    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bl77\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-kube-api-access-8bl77\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:19.740034 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.740026    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ee77f29-b901-4674-a7c5-50380d1100e6-ca-trust-extracted\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:19.740034 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.740036    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-certificates\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:19.740270 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.740045    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ee77f29-b901-4674-a7c5-50380d1100e6-trusted-ca\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:19.740270 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.740054    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-bound-sa-token\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:19.740270 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.740061    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-installation-pull-secrets\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:19.740270 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.740069    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ee77f29-b901-4674-a7c5-50380d1100e6-registry-tls\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:19.740270 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:19.740079    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ee77f29-b901-4674-a7c5-50380d1100e6-image-registry-private-configuration\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:20.252248 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:20.252216    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm"
Jul 12 15:15:20.252248 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:20.252220    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65b85f644b-ss8lm" event={"ID":"9ee77f29-b901-4674-a7c5-50380d1100e6","Type":"ContainerDied","Data":"f2c52452e1affe6ec30338d12973938206c6c1f62ff6e284c59b3af6edab3103"}
Jul 12 15:15:20.252753 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:20.252281    2578 scope.go:117] "RemoveContainer" containerID="040407103acb9bd5724ddbfe7fe97f8189d10b634154f09b4d60ae613a37c4ed"
Jul 12 15:15:20.267941 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:20.267917    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65b85f644b-ss8lm"]
Jul 12 15:15:20.271198 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:20.271172    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65b85f644b-ss8lm"]
Jul 12 15:15:21.654470 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.654406    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c4b9b766d-4h6dr" podUID="d1758264-b8b2-42ef-9c51-35929219892a" containerName="console" containerID="cri-o://f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79" gracePeriod=15
Jul 12 15:15:21.904901 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.904837    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:15:21.957485 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.957446    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5r5t\" (UniqueName: \"kubernetes.io/projected/d1758264-b8b2-42ef-9c51-35929219892a-kube-api-access-x5r5t\") pod \"d1758264-b8b2-42ef-9c51-35929219892a\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") "
Jul 12 15:15:21.957660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.957508    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-console-config\") pod \"d1758264-b8b2-42ef-9c51-35929219892a\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") "
Jul 12 15:15:21.957660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.957545    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-oauth-serving-cert\") pod \"d1758264-b8b2-42ef-9c51-35929219892a\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") "
Jul 12 15:15:21.957660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.957594    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-serving-cert\") pod \"d1758264-b8b2-42ef-9c51-35929219892a\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") "
Jul 12 15:15:21.957660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.957627    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-service-ca\") pod \"d1758264-b8b2-42ef-9c51-35929219892a\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") "
Jul 12 15:15:21.957855 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.957666    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-oauth-config\") pod \"d1758264-b8b2-42ef-9c51-35929219892a\" (UID: \"d1758264-b8b2-42ef-9c51-35929219892a\") "
Jul 12 15:15:21.957989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.957952    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d1758264-b8b2-42ef-9c51-35929219892a" (UID: "d1758264-b8b2-42ef-9c51-35929219892a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:15:21.957989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.957971    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-console-config" (OuterVolumeSpecName: "console-config") pod "d1758264-b8b2-42ef-9c51-35929219892a" (UID: "d1758264-b8b2-42ef-9c51-35929219892a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:15:21.958167 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.958072    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-service-ca" (OuterVolumeSpecName: "service-ca") pod "d1758264-b8b2-42ef-9c51-35929219892a" (UID: "d1758264-b8b2-42ef-9c51-35929219892a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:15:21.959934 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.959906    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d1758264-b8b2-42ef-9c51-35929219892a" (UID: "d1758264-b8b2-42ef-9c51-35929219892a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:15:21.959934 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.959921    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d1758264-b8b2-42ef-9c51-35929219892a" (UID: "d1758264-b8b2-42ef-9c51-35929219892a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:15:21.960062 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:21.959994    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1758264-b8b2-42ef-9c51-35929219892a-kube-api-access-x5r5t" (OuterVolumeSpecName: "kube-api-access-x5r5t") pod "d1758264-b8b2-42ef-9c51-35929219892a" (UID: "d1758264-b8b2-42ef-9c51-35929219892a"). InnerVolumeSpecName "kube-api-access-x5r5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:15:22.059229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.059190    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5r5t\" (UniqueName: \"kubernetes.io/projected/d1758264-b8b2-42ef-9c51-35929219892a-kube-api-access-x5r5t\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:22.059229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.059221    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-console-config\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:22.059229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.059235    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-oauth-serving-cert\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:22.059454 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.059247    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-serving-cert\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:22.059454 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.059258    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1758264-b8b2-42ef-9c51-35929219892a-service-ca\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:22.059454 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.059269    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1758264-b8b2-42ef-9c51-35929219892a-console-oauth-config\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:15:22.155274 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.155177    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ee77f29-b901-4674-a7c5-50380d1100e6" path="/var/lib/kubelet/pods/9ee77f29-b901-4674-a7c5-50380d1100e6/volumes"
Jul 12 15:15:22.263628 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.263596    2578 generic.go:356] "Generic (PLEG): container finished" podID="d1758264-b8b2-42ef-9c51-35929219892a" containerID="f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79" exitCode=0
Jul 12 15:15:22.263836 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.263634    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4b9b766d-4h6dr"
Jul 12 15:15:22.263836 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.263634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4b9b766d-4h6dr" event={"ID":"d1758264-b8b2-42ef-9c51-35929219892a","Type":"ContainerDied","Data":"f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79"}
Jul 12 15:15:22.263836 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.263758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4b9b766d-4h6dr" event={"ID":"d1758264-b8b2-42ef-9c51-35929219892a","Type":"ContainerDied","Data":"a9e4519b92a339c85a7fc1cbfb188401cdf0b12e8566d4c4a4bf72a338af6596"}
Jul 12 15:15:22.263836 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.263780    2578 scope.go:117] "RemoveContainer" containerID="f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79"
Jul 12 15:15:22.272779 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.272758    2578 scope.go:117] "RemoveContainer" containerID="f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79"
Jul 12 15:15:22.273080 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:15:22.273059    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79\": container with ID starting with f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79 not found: ID does not exist" containerID="f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79"
Jul 12 15:15:22.273148 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.273090    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79"} err="failed to get container status \"f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79\": rpc error: code = NotFound desc = could not find container \"f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79\": container with ID starting with f1feb7237d06f9d3170c5e8322f0124a477890a57d593bb9fde4c7b72b528e79 not found: ID does not exist"
Jul 12 15:15:22.278461 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.278437    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c4b9b766d-4h6dr"]
Jul 12 15:15:22.280669 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:22.280648    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c4b9b766d-4h6dr"]
Jul 12 15:15:24.158001 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:24.155447    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1758264-b8b2-42ef-9c51-35929219892a" path="/var/lib/kubelet/pods/d1758264-b8b2-42ef-9c51-35929219892a/volumes"
Jul 12 15:15:45.507560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:45.507525    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:15:45.530435 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:45.530405    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:15:46.376700 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:46.376676    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:15:56.267820 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:56.267756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:15:56.270039 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:56.270019    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc058fa-537e-49c8-a4b9-79fe8986f083-metrics-certs\") pod \"network-metrics-daemon-6vsjp\" (UID: \"dcc058fa-537e-49c8-a4b9-79fe8986f083\") " pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:15:56.453495 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:56.453466    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mn9r4\""
Jul 12 15:15:56.461454 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:56.461436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vsjp"
Jul 12 15:15:56.588473 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:56.588285    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6vsjp"]
Jul 12 15:15:56.591183 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:15:56.591158    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc058fa_537e_49c8_a4b9_79fe8986f083.slice/crio-bed9b8b1351e894c5071e04ebb860d4c43fb116c5461eb932ebc12e290950912 WatchSource:0}: Error finding container bed9b8b1351e894c5071e04ebb860d4c43fb116c5461eb932ebc12e290950912: Status 404 returned error can't find the container with id bed9b8b1351e894c5071e04ebb860d4c43fb116c5461eb932ebc12e290950912
Jul 12 15:15:57.405055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:57.405007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6vsjp" event={"ID":"dcc058fa-537e-49c8-a4b9-79fe8986f083","Type":"ContainerStarted","Data":"bed9b8b1351e894c5071e04ebb860d4c43fb116c5461eb932ebc12e290950912"}
Jul 12 15:15:58.410387 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:58.410350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6vsjp" event={"ID":"dcc058fa-537e-49c8-a4b9-79fe8986f083","Type":"ContainerStarted","Data":"fef1561ad436d3218cc841047f4e86d7136b6b6a53ee8e45c1ec4f578b05d672"}
Jul 12 15:15:58.410771 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:58.410413    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6vsjp" event={"ID":"dcc058fa-537e-49c8-a4b9-79fe8986f083","Type":"ContainerStarted","Data":"013d5c0180c815430f3844937b09b675475d347963af1c77e67aaa0dde1155fc"}
Jul 12 15:15:58.426789 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:58.426737    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6vsjp" podStartSLOduration=375.290856241 podStartE2EDuration="6m16.426722468s" totalImagesPullingTime="1.135866227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:15:58.425492832 +0000 UTC m=+376.793093694" watchObservedRunningTime="2026-07-12 15:15:58.426722468 +0000 UTC m=+376.794323334"
Jul 12 15:15:59.268496 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.268455    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:15:59.268747 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.268727    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="alertmanager" containerID="cri-o://21e185f360808c0b60172aa8c6e34b2326b0ac3d1c1743aa485546291e49691d" gracePeriod=120
Jul 12 15:15:59.268845 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.268795    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy-metric" containerID="cri-o://e4e5009c6292653bbec7bd9b43dc15cbc1b8bbef9201e0f00706f87fb6141d34" gracePeriod=120
Jul 12 15:15:59.268845 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.268832    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy-web" containerID="cri-o://e191ecae29f3c9f3a5abd92cbb95ccc8fdfda4d6ef8afbebc6ab7b96e9342b50" gracePeriod=120
Jul 12 15:15:59.269039 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.268867    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="config-reloader" containerID="cri-o://f08136ea1cdf509b19b493baf1958fcfc004d337be4d359cbc8bad6ead733b80" gracePeriod=120
Jul 12 15:15:59.269039 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.268915    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="prom-label-proxy" containerID="cri-o://bad83f3aaa5554a95e54f0bc158c13c75b212d8493e756f9944681de84dbfd52" gracePeriod=120
Jul 12 15:15:59.269039 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.268877    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy" containerID="cri-o://9d03d77f47746eed2e00af10ab16891870b1ff3502b8d1731ce808968cc2dca5" gracePeriod=120
Jul 12 15:15:59.417385 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.417345    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerID="bad83f3aaa5554a95e54f0bc158c13c75b212d8493e756f9944681de84dbfd52" exitCode=0
Jul 12 15:15:59.417385 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.417371    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerID="9d03d77f47746eed2e00af10ab16891870b1ff3502b8d1731ce808968cc2dca5" exitCode=0
Jul 12 15:15:59.417385 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.417379    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerID="f08136ea1cdf509b19b493baf1958fcfc004d337be4d359cbc8bad6ead733b80" exitCode=0
Jul 12 15:15:59.417385 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.417388    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerID="21e185f360808c0b60172aa8c6e34b2326b0ac3d1c1743aa485546291e49691d" exitCode=0
Jul 12 15:15:59.417972 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.417410    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerDied","Data":"bad83f3aaa5554a95e54f0bc158c13c75b212d8493e756f9944681de84dbfd52"}
Jul 12 15:15:59.417972 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.417475    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerDied","Data":"9d03d77f47746eed2e00af10ab16891870b1ff3502b8d1731ce808968cc2dca5"}
Jul 12 15:15:59.417972 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.417504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerDied","Data":"f08136ea1cdf509b19b493baf1958fcfc004d337be4d359cbc8bad6ead733b80"}
Jul 12 15:15:59.417972 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:15:59.417530    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerDied","Data":"21e185f360808c0b60172aa8c6e34b2326b0ac3d1c1743aa485546291e49691d"}
Jul 12 15:16:00.425765 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.425730    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerID="e4e5009c6292653bbec7bd9b43dc15cbc1b8bbef9201e0f00706f87fb6141d34" exitCode=0
Jul 12 15:16:00.425765 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.425755    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerID="e191ecae29f3c9f3a5abd92cbb95ccc8fdfda4d6ef8afbebc6ab7b96e9342b50" exitCode=0
Jul 12 15:16:00.426263 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.425774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerDied","Data":"e4e5009c6292653bbec7bd9b43dc15cbc1b8bbef9201e0f00706f87fb6141d34"}
Jul 12 15:16:00.426263 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.425836    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerDied","Data":"e191ecae29f3c9f3a5abd92cbb95ccc8fdfda4d6ef8afbebc6ab7b96e9342b50"}
Jul 12 15:16:00.519008 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.518981    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:00.707439 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707357    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-out\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707439 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707390    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-tls-assets\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707439 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707419    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-web\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707445    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-trusted-ca-bundle\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707460    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-main-db\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707476    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707501    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-volume\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707526    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707550    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-web-config\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707574    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjkqx\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-kube-api-access-gjkqx\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707602    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-main-tls\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707642    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-cluster-tls-config\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.707679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707681    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-metrics-client-ca\") pod \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\" (UID: \"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1\") "
Jul 12 15:16:00.708201 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707849    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:00.708201 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.707974    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.708305 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.708200    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:00.708305 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.708250    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:16:00.710497 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.710456    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-out" (OuterVolumeSpecName: "config-out") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:16:00.710497 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.710475    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:00.710772 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.710748    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:16:00.710918 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.710904    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-volume" (OuterVolumeSpecName: "config-volume") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:00.711019 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.710998    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-kube-api-access-gjkqx" (OuterVolumeSpecName: "kube-api-access-gjkqx") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "kube-api-access-gjkqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:16:00.711146 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.711126    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:00.711277 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.711259    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:00.711983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.711953    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:00.715205 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.715186    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:00.722133 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.722114    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-web-config" (OuterVolumeSpecName: "web-config") pod "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" (UID: "e9eee1b1-cfe9-4d25-9be8-bc2805e359f1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:00.809222 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809184    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-out\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809222 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809214    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-tls-assets\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809222 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809224    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809222 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809233    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-alertmanager-main-db\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809469 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809243    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809469 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809252    2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-config-volume\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809469 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809261    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809469 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809269    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-web-config\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809469 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809276    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjkqx\" (UniqueName: \"kubernetes.io/projected/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-kube-api-access-gjkqx\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809469 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809284    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-secret-alertmanager-main-tls\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809469 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809292    2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-cluster-tls-config\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:00.809469 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:00.809300    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1-metrics-client-ca\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:01.432175 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.432143    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9eee1b1-cfe9-4d25-9be8-bc2805e359f1","Type":"ContainerDied","Data":"5c1892a854bdaa9f5cc5feef8ea891dc53599bb4911caebc301f915044a9f373"}
Jul 12 15:16:01.432589 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.432201    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.432589 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.432219    2578 scope.go:117] "RemoveContainer" containerID="bad83f3aaa5554a95e54f0bc158c13c75b212d8493e756f9944681de84dbfd52"
Jul 12 15:16:01.440159 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.440122    2578 scope.go:117] "RemoveContainer" containerID="e4e5009c6292653bbec7bd9b43dc15cbc1b8bbef9201e0f00706f87fb6141d34"
Jul 12 15:16:01.447352 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.447334    2578 scope.go:117] "RemoveContainer" containerID="9d03d77f47746eed2e00af10ab16891870b1ff3502b8d1731ce808968cc2dca5"
Jul 12 15:16:01.454507 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.454486    2578 scope.go:117] "RemoveContainer" containerID="e191ecae29f3c9f3a5abd92cbb95ccc8fdfda4d6ef8afbebc6ab7b96e9342b50"
Jul 12 15:16:01.454571 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.454548    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:16:01.458224 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.458203    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:16:01.461648 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.461629    2578 scope.go:117] "RemoveContainer" containerID="f08136ea1cdf509b19b493baf1958fcfc004d337be4d359cbc8bad6ead733b80"
Jul 12 15:16:01.468372 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.468356    2578 scope.go:117] "RemoveContainer" containerID="21e185f360808c0b60172aa8c6e34b2326b0ac3d1c1743aa485546291e49691d"
Jul 12 15:16:01.475468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.475452    2578 scope.go:117] "RemoveContainer" containerID="aa67b9c2939d450addec9a98c160c3e3ac13923199eb0678e637d118f7675731"
Jul 12 15:16:01.481091 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481072    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:16:01.481394 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481381    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy-metric"
Jul 12 15:16:01.481447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481396    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy-metric"
Jul 12 15:16:01.481447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481406    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy-web"
Jul 12 15:16:01.481447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481412    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy-web"
Jul 12 15:16:01.481447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481425    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="init-config-reloader"
Jul 12 15:16:01.481447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481430    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="init-config-reloader"
Jul 12 15:16:01.481447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481438    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1758264-b8b2-42ef-9c51-35929219892a" containerName="console"
Jul 12 15:16:01.481447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481443    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1758264-b8b2-42ef-9c51-35929219892a" containerName="console"
Jul 12 15:16:01.481447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481448    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="alertmanager"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481454    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="alertmanager"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481460    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="prom-label-proxy"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481465    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="prom-label-proxy"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481473    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee77f29-b901-4674-a7c5-50380d1100e6" containerName="registry"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481478    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee77f29-b901-4674-a7c5-50380d1100e6" containerName="registry"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481487    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="config-reloader"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481492    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="config-reloader"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481501    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481505    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481543    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy-metric"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481550    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="alertmanager"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481555    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481561    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="prom-label-proxy"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481566    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="kube-rbac-proxy-web"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481571    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1758264-b8b2-42ef-9c51-35929219892a" containerName="console"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481575    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" containerName="config-reloader"
Jul 12 15:16:01.481687 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.481582    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ee77f29-b901-4674-a7c5-50380d1100e6" containerName="registry"
Jul 12 15:16:01.486354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.486338    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.489073 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489053    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 12 15:16:01.489193 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489110    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 12 15:16:01.489193 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489172    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 12 15:16:01.489305 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489252    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 12 15:16:01.489305 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489265    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 12 15:16:01.489398 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489390    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vvgds\""
Jul 12 15:16:01.489546 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489533    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 12 15:16:01.489622 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489544    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 12 15:16:01.489622 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.489562    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 12 15:16:01.499226 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.499058    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:16:01.500356 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.500334    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 12 15:16:01.615631 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.615631 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.615631 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.615908 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615701    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.615908 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.615908 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615782    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.615908 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-config-out\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.615908 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615888    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.615908 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cntk7\" (UniqueName: \"kubernetes.io/projected/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-kube-api-access-cntk7\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.616128 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-config-volume\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.616128 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.616128 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615968    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.616128 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.615995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-web-config\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716789 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716755    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-config-volume\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716861    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-web-config\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716878    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716955    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.716983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.716980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.717479 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.717413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.717773 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.717642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.717773 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.717700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-config-out\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.718284 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.717977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.718284 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.717990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.718284 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.718045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cntk7\" (UniqueName: \"kubernetes.io/projected/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-kube-api-access-cntk7\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.719887 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.719867    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.719977 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.719888    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-web-config\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.720040 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.719998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-config-volume\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.720126 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.720081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.720564 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.720538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.720651 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.720612    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.720727 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.720690    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.720789 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.720778    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.720876 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.720837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.721718 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.721703    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-config-out\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.725479 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.725456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cntk7\" (UniqueName: \"kubernetes.io/projected/573cd437-a92f-41dd-bf57-d5f61d7ea8b1-kube-api-access-cntk7\") pod \"alertmanager-main-0\" (UID: \"573cd437-a92f-41dd-bf57-d5f61d7ea8b1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.801416 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.801389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 15:16:01.930949 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:01.930926    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 15:16:01.933448 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:16:01.933421    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573cd437_a92f_41dd_bf57_d5f61d7ea8b1.slice/crio-96cbecc9c644d88ed6822a0f1347009a7a946695e305278ae248a73f3c6f7973 WatchSource:0}: Error finding container 96cbecc9c644d88ed6822a0f1347009a7a946695e305278ae248a73f3c6f7973: Status 404 returned error can't find the container with id 96cbecc9c644d88ed6822a0f1347009a7a946695e305278ae248a73f3c6f7973
Jul 12 15:16:02.154909 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:02.154835    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9eee1b1-cfe9-4d25-9be8-bc2805e359f1" path="/var/lib/kubelet/pods/e9eee1b1-cfe9-4d25-9be8-bc2805e359f1/volumes"
Jul 12 15:16:02.438078 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:02.437997    2578 generic.go:356] "Generic (PLEG): container finished" podID="573cd437-a92f-41dd-bf57-d5f61d7ea8b1" containerID="a6d116f433a819f296d35bb4e5dba520bc2fdccf71c9038604370c93255aed05" exitCode=0
Jul 12 15:16:02.438078 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:02.438033    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"573cd437-a92f-41dd-bf57-d5f61d7ea8b1","Type":"ContainerDied","Data":"a6d116f433a819f296d35bb4e5dba520bc2fdccf71c9038604370c93255aed05"}
Jul 12 15:16:02.438450 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:02.438082    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"573cd437-a92f-41dd-bf57-d5f61d7ea8b1","Type":"ContainerStarted","Data":"96cbecc9c644d88ed6822a0f1347009a7a946695e305278ae248a73f3c6f7973"}
Jul 12 15:16:03.279470 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.279431    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"]
Jul 12 15:16:03.283028 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.283008    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.285258 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.285231    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 12 15:16:03.285424 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.285405    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 12 15:16:03.285505 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.285462    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 12 15:16:03.285847 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.285827    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2tcxf\""
Jul 12 15:16:03.285941 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.285853    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 12 15:16:03.285989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.285934    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 12 15:16:03.289357 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.289340    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 12 15:16:03.294251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.294229    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"]
Jul 12 15:16:03.332093 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.332056    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvwgc\" (UniqueName: \"kubernetes.io/projected/e705a0a9-f163-41de-8976-b9958cda7a89-kube-api-access-gvwgc\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.332273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.332128    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-secret-telemeter-client\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.332273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.332150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-metrics-client-ca\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.332273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.332178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.332273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.332197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-telemeter-client-tls\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.332468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.332285    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-federate-client-tls\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.332468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.332321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.332468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.332346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-serving-certs-ca-bundle\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.433081 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.433041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-secret-telemeter-client\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.433081 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.433084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-metrics-client-ca\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.433321 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.433115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.433321 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.433136    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-telemeter-client-tls\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.433321 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.433159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-federate-client-tls\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.433321 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.433177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.433321 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.433191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-serving-certs-ca-bundle\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.433321 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.433244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvwgc\" (UniqueName: \"kubernetes.io/projected/e705a0a9-f163-41de-8976-b9958cda7a89-kube-api-access-gvwgc\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.434081 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.434054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-metrics-client-ca\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.434203 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.434109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-serving-certs-ca-bundle\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.434355 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.434322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e705a0a9-f163-41de-8976-b9958cda7a89-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.435592 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.435570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.435703 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.435661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-secret-telemeter-client\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.435869 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.435847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-telemeter-client-tls\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.435919 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.435856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e705a0a9-f163-41de-8976-b9958cda7a89-federate-client-tls\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.440969 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.440940    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvwgc\" (UniqueName: \"kubernetes.io/projected/e705a0a9-f163-41de-8976-b9958cda7a89-kube-api-access-gvwgc\") pod \"telemeter-client-b6f595dcc-mqrs8\" (UID: \"e705a0a9-f163-41de-8976-b9958cda7a89\") " pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.446928 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.446904    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"573cd437-a92f-41dd-bf57-d5f61d7ea8b1","Type":"ContainerStarted","Data":"348328c0329132276b81a129fde5417946c382348905709834f43afef99c8a41"}
Jul 12 15:16:03.447034 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.446988    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"573cd437-a92f-41dd-bf57-d5f61d7ea8b1","Type":"ContainerStarted","Data":"b260b81cfbc4a7a3a2bb5484799332a5916e8f282cf7c52c80136bdbe6d8fb6c"}
Jul 12 15:16:03.447034 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.447015    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"573cd437-a92f-41dd-bf57-d5f61d7ea8b1","Type":"ContainerStarted","Data":"6a07f2c99cf311fd536c69e51be63c48554e178523717fd0d51d8e1a9dcfb94f"}
Jul 12 15:16:03.447126 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.447041    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"573cd437-a92f-41dd-bf57-d5f61d7ea8b1","Type":"ContainerStarted","Data":"a5e5ab4a7529d79dede29539c8388f11950689c1812c75a29f71f01119cdf5bd"}
Jul 12 15:16:03.447126 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.447068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"573cd437-a92f-41dd-bf57-d5f61d7ea8b1","Type":"ContainerStarted","Data":"200f49849c21f380b2b4bca43425ce0657fb8ffa0ea3ec99a40260a3ad407287"}
Jul 12 15:16:03.447126 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.447090    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"573cd437-a92f-41dd-bf57-d5f61d7ea8b1","Type":"ContainerStarted","Data":"31c1cd0716054d6b439418394e71bacab1dcac0db0d0651dbeacae073afc9136"}
Jul 12 15:16:03.469864 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.469356    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.455232958 podStartE2EDuration="2.46934049s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.107532ms" isStatefulPod=false podCreationTimestamp="2026-07-12 15:16:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:16:03.468395241 +0000 UTC m=+381.835996103" watchObservedRunningTime="2026-07-12 15:16:03.46934049 +0000 UTC m=+381.836941337"
Jul 12 15:16:03.563061 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.562981    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:16:03.563293 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.563252    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="prometheus" containerID="cri-o://80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443" gracePeriod=600
Jul 12 15:16:03.563434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.563275    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy" containerID="cri-o://a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f" gracePeriod=600
Jul 12 15:16:03.563434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.563314    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy-web" containerID="cri-o://73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b" gracePeriod=600
Jul 12 15:16:03.563434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.563319    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="thanos-sidecar" containerID="cri-o://3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f" gracePeriod=600
Jul 12 15:16:03.563434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.563336    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="config-reloader" containerID="cri-o://6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1" gracePeriod=600
Jul 12 15:16:03.563434 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.563291    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy-thanos" containerID="cri-o://adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec" gracePeriod=600
Jul 12 15:16:03.592184 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.592155    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"
Jul 12 15:16:03.732255 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.732078    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b6f595dcc-mqrs8"]
Jul 12 15:16:03.734837 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:16:03.734792    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode705a0a9_f163_41de_8976_b9958cda7a89.slice/crio-76411cc9bad955ddc79714304874caeabead4e6723b085bcfff574cbf2b422dc WatchSource:0}: Error finding container 76411cc9bad955ddc79714304874caeabead4e6723b085bcfff574cbf2b422dc: Status 404 returned error can't find the container with id 76411cc9bad955ddc79714304874caeabead4e6723b085bcfff574cbf2b422dc
Jul 12 15:16:03.813229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.813178    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:03.837294 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837266    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-trusted-ca-bundle\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837301    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhr6c\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-kube-api-access-vhr6c\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837325    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-serving-certs-ca-bundle\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837345    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-db\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837376    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-tls\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837393    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-thanos-prometheus-http-client-file\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837421    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-1\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837448    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-kubelet-serving-ca-bundle\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837472    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837512    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-2\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837550    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config-out\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837566    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-metrics-client-certs\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837592    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-kube-rbac-proxy\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837616    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-web-config\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837671    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.837720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837715    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-grpc-tls\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837747    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-metrics-client-ca\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837787    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837793    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837832    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-tls-assets\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837832    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837862    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-0\") pod \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\" (UID: \"cc4e00ac-07b0-4f01-a3ad-d36c7281e511\") "
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.837923    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.838133    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.838150    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.838210 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.838165    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.838675 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.838268    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:03.838675 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.838577    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:03.838675 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.838638    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:16:03.839376 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.839347    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:03.839762 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.839734    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:16:03.839923 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.839896    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-kube-api-access-vhr6c" (OuterVolumeSpecName: "kube-api-access-vhr6c") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "kube-api-access-vhr6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:16:03.841527 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.841379    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.841527 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.841480    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.841527 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.841501    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.842183 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.842157    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.843105 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.843077    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:16:03.843181 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.843088    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.843181 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.843134    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config" (OuterVolumeSpecName: "config") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.843181 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.843160    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.843833 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.843792    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config-out" (OuterVolumeSpecName: "config-out") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:16:03.844517 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.844496    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.853488 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.853461    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-web-config" (OuterVolumeSpecName: "web-config") pod "cc4e00ac-07b0-4f01-a3ad-d36c7281e511" (UID: "cc4e00ac-07b0-4f01-a3ad-d36c7281e511"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:16:03.938534 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938500    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938534 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938529    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938534 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938539    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938547    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config-out\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938556    2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-metrics-client-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938565    2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-kube-rbac-proxy\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938573    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-web-config\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938580    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938588    2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-grpc-tls\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938597    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-configmap-metrics-client-ca\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938604    2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-config\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938612    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-tls-assets\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938619    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938628    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhr6c\" (UniqueName: \"kubernetes.io/projected/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-kube-api-access-vhr6c\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938636    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-prometheus-k8s-db\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938645    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:03.938787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:03.938653    2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cc4e00ac-07b0-4f01-a3ad-d36c7281e511-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:16:04.454893 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.454787    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8" event={"ID":"e705a0a9-f163-41de-8976-b9958cda7a89","Type":"ContainerStarted","Data":"76411cc9bad955ddc79714304874caeabead4e6723b085bcfff574cbf2b422dc"}
Jul 12 15:16:04.458349 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458320    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec" exitCode=0
Jul 12 15:16:04.458349 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458350    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f" exitCode=0
Jul 12 15:16:04.458542 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458360    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b" exitCode=0
Jul 12 15:16:04.458542 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458371    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f" exitCode=0
Jul 12 15:16:04.458542 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458380    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1" exitCode=0
Jul 12 15:16:04.458542 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458388    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443" exitCode=0
Jul 12 15:16:04.458542 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458400    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.458542 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458404    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerDied","Data":"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"}
Jul 12 15:16:04.458542 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458478    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerDied","Data":"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"}
Jul 12 15:16:04.458542 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerDied","Data":"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"}
Jul 12 15:16:04.458917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458555    2578 scope.go:117] "RemoveContainer" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"
Jul 12 15:16:04.458917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458562    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerDied","Data":"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"}
Jul 12 15:16:04.458917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458604    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerDied","Data":"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"}
Jul 12 15:16:04.458917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458644    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerDied","Data":"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"}
Jul 12 15:16:04.458917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.458685    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc4e00ac-07b0-4f01-a3ad-d36c7281e511","Type":"ContainerDied","Data":"302a5d74fe194a2968b9b1873588c9ab691e72fc9a9177ea825f62a094f5157c"}
Jul 12 15:16:04.467662 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.467643    2578 scope.go:117] "RemoveContainer" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"
Jul 12 15:16:04.476261 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.476241    2578 scope.go:117] "RemoveContainer" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"
Jul 12 15:16:04.481982 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.481958    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:16:04.485332 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.485313    2578 scope.go:117] "RemoveContainer" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"
Jul 12 15:16:04.487984 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.487960    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:16:04.493413 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.493394    2578 scope.go:117] "RemoveContainer" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"
Jul 12 15:16:04.500795 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.500765    2578 scope.go:117] "RemoveContainer" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"
Jul 12 15:16:04.509139 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.509123    2578 scope.go:117] "RemoveContainer" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"
Jul 12 15:16:04.512267 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512248    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:16:04.512544 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512533    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="prometheus"
Jul 12 15:16:04.512584 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512546    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="prometheus"
Jul 12 15:16:04.512584 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512556    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy"
Jul 12 15:16:04.512584 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512562    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy"
Jul 12 15:16:04.512584 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512580    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="init-config-reloader"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512589    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="init-config-reloader"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512599    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="thanos-sidecar"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512607    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="thanos-sidecar"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512617    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy-web"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512625    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy-web"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512638    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="config-reloader"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512645    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="config-reloader"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512658    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy-thanos"
Jul 12 15:16:04.512706 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512666    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy-thanos"
Jul 12 15:16:04.512995 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512750    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="prometheus"
Jul 12 15:16:04.512995 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512762    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="thanos-sidecar"
Jul 12 15:16:04.512995 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512775    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="config-reloader"
Jul 12 15:16:04.512995 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512785    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy"
Jul 12 15:16:04.512995 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512795    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy-thanos"
Jul 12 15:16:04.512995 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.512823    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" containerName="kube-rbac-proxy-web"
Jul 12 15:16:04.518292 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.518278    2578 scope.go:117] "RemoveContainer" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"
Jul 12 15:16:04.518544 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:16:04.518525    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": container with ID starting with adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec not found: ID does not exist" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"
Jul 12 15:16:04.518605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.518551    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"} err="failed to get container status \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": rpc error: code = NotFound desc = could not find container \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": container with ID starting with adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec not found: ID does not exist"
Jul 12 15:16:04.518605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.518569    2578 scope.go:117] "RemoveContainer" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"
Jul 12 15:16:04.518783 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:16:04.518765    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": container with ID starting with a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f not found: ID does not exist" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"
Jul 12 15:16:04.518845 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.518788    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"} err="failed to get container status \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": rpc error: code = NotFound desc = could not find container \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": container with ID starting with a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f not found: ID does not exist"
Jul 12 15:16:04.518845 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.518824    2578 scope.go:117] "RemoveContainer" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"
Jul 12 15:16:04.519023 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:16:04.519006    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": container with ID starting with 73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b not found: ID does not exist" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"
Jul 12 15:16:04.519107 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.519023    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"} err="failed to get container status \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": rpc error: code = NotFound desc = could not find container \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": container with ID starting with 73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b not found: ID does not exist"
Jul 12 15:16:04.519107 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.519035    2578 scope.go:117] "RemoveContainer" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"
Jul 12 15:16:04.519249 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.519234    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.519337 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:16:04.519232    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": container with ID starting with 3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f not found: ID does not exist" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"
Jul 12 15:16:04.519381 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.519350    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"} err="failed to get container status \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": rpc error: code = NotFound desc = could not find container \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": container with ID starting with 3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f not found: ID does not exist"
Jul 12 15:16:04.519381 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.519369    2578 scope.go:117] "RemoveContainer" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"
Jul 12 15:16:04.519855 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:16:04.519828    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": container with ID starting with 6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1 not found: ID does not exist" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"
Jul 12 15:16:04.519933 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.519863    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"} err="failed to get container status \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": rpc error: code = NotFound desc = could not find container \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": container with ID starting with 6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1 not found: ID does not exist"
Jul 12 15:16:04.519933 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.519884    2578 scope.go:117] "RemoveContainer" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"
Jul 12 15:16:04.520148 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:16:04.520128    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": container with ID starting with 80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443 not found: ID does not exist" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"
Jul 12 15:16:04.520193 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.520153    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"} err="failed to get container status \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": rpc error: code = NotFound desc = could not find container \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": container with ID starting with 80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443 not found: ID does not exist"
Jul 12 15:16:04.520193 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.520166    2578 scope.go:117] "RemoveContainer" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"
Jul 12 15:16:04.520379 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:16:04.520358    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": container with ID starting with f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231 not found: ID does not exist" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"
Jul 12 15:16:04.520425 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.520390    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"} err="failed to get container status \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": rpc error: code = NotFound desc = could not find container \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": container with ID starting with f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231 not found: ID does not exist"
Jul 12 15:16:04.520425 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.520410    2578 scope.go:117] "RemoveContainer" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"
Jul 12 15:16:04.520642 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.520625    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"} err="failed to get container status \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": rpc error: code = NotFound desc = could not find container \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": container with ID starting with adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec not found: ID does not exist"
Jul 12 15:16:04.520720 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.520642    2578 scope.go:117] "RemoveContainer" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"
Jul 12 15:16:04.520874 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.520859    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"} err="failed to get container status \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": rpc error: code = NotFound desc = could not find container \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": container with ID starting with a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f not found: ID does not exist"
Jul 12 15:16:04.520934 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.520875    2578 scope.go:117] "RemoveContainer" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"
Jul 12 15:16:04.521088 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521072    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"} err="failed to get container status \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": rpc error: code = NotFound desc = could not find container \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": container with ID starting with 73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b not found: ID does not exist"
Jul 12 15:16:04.521162 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521090    2578 scope.go:117] "RemoveContainer" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"
Jul 12 15:16:04.521328 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521306    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"} err="failed to get container status \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": rpc error: code = NotFound desc = could not find container \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": container with ID starting with 3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f not found: ID does not exist"
Jul 12 15:16:04.521443 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521330    2578 scope.go:117] "RemoveContainer" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"
Jul 12 15:16:04.521593 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521547    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"} err="failed to get container status \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": rpc error: code = NotFound desc = could not find container \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": container with ID starting with 6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1 not found: ID does not exist"
Jul 12 15:16:04.521593 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521566    2578 scope.go:117] "RemoveContainer" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"
Jul 12 15:16:04.521794 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521752    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"} err="failed to get container status \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": rpc error: code = NotFound desc = could not find container \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": container with ID starting with 80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443 not found: ID does not exist"
Jul 12 15:16:04.521794 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521775    2578 scope.go:117] "RemoveContainer" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"
Jul 12 15:16:04.521969 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521927    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1pvt3j211dbgi\""
Jul 12 15:16:04.521969 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.521930    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 12 15:16:04.522087 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522065    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"} err="failed to get container status \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": rpc error: code = NotFound desc = could not find container \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": container with ID starting with f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231 not found: ID does not exist"
Jul 12 15:16:04.522186 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522091    2578 scope.go:117] "RemoveContainer" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"
Jul 12 15:16:04.522304 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522246    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 12 15:16:04.522304 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522246    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 12 15:16:04.522304 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522266    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 12 15:16:04.522304 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522285    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 12 15:16:04.522304 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522296    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"} err="failed to get container status \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": rpc error: code = NotFound desc = could not find container \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": container with ID starting with adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec not found: ID does not exist"
Jul 12 15:16:04.522304 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522317    2578 scope.go:117] "RemoveContainer" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"
Jul 12 15:16:04.522746 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522367    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 12 15:16:04.522746 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522403    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 12 15:16:04.522746 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522609    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"} err="failed to get container status \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": rpc error: code = NotFound desc = could not find container \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": container with ID starting with a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f not found: ID does not exist"
Jul 12 15:16:04.522746 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.522628    2578 scope.go:117] "RemoveContainer" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"
Jul 12 15:16:04.523723 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.523072    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 12 15:16:04.523723 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.523142    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 12 15:16:04.523723 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.523254    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 12 15:16:04.523723 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.523489    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qr24q\""
Jul 12 15:16:04.523723 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.523500    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 12 15:16:04.524138 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.523849    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"} err="failed to get container status \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": rpc error: code = NotFound desc = could not find container \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": container with ID starting with 73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b not found: ID does not exist"
Jul 12 15:16:04.524138 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.523882    2578 scope.go:117] "RemoveContainer" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"
Jul 12 15:16:04.524138 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.523981    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 12 15:16:04.525055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.524554    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"} err="failed to get container status \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": rpc error: code = NotFound desc = could not find container \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": container with ID starting with 3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f not found: ID does not exist"
Jul 12 15:16:04.525055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.524593    2578 scope.go:117] "RemoveContainer" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"
Jul 12 15:16:04.525055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.524871    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 12 15:16:04.525593 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.525376    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"} err="failed to get container status \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": rpc error: code = NotFound desc = could not find container \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": container with ID starting with 6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1 not found: ID does not exist"
Jul 12 15:16:04.525593 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.525402    2578 scope.go:117] "RemoveContainer" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"
Jul 12 15:16:04.525735 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.525640    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"} err="failed to get container status \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": rpc error: code = NotFound desc = could not find container \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": container with ID starting with 80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443 not found: ID does not exist"
Jul 12 15:16:04.525735 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.525703    2578 scope.go:117] "RemoveContainer" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"
Jul 12 15:16:04.526059 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.526033    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"} err="failed to get container status \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": rpc error: code = NotFound desc = could not find container \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": container with ID starting with f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231 not found: ID does not exist"
Jul 12 15:16:04.526144 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.526060    2578 scope.go:117] "RemoveContainer" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"
Jul 12 15:16:04.526408 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.526386    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"} err="failed to get container status \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": rpc error: code = NotFound desc = could not find container \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": container with ID starting with adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec not found: ID does not exist"
Jul 12 15:16:04.526486 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.526410    2578 scope.go:117] "RemoveContainer" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"
Jul 12 15:16:04.526889 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.526734    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"} err="failed to get container status \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": rpc error: code = NotFound desc = could not find container \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": container with ID starting with a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f not found: ID does not exist"
Jul 12 15:16:04.526889 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.526762    2578 scope.go:117] "RemoveContainer" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"
Jul 12 15:16:04.531582 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.531561    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 12 15:16:04.531704 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.531683    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 12 15:16:04.532733 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.532449    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"} err="failed to get container status \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": rpc error: code = NotFound desc = could not find container \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": container with ID starting with 73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b not found: ID does not exist"
Jul 12 15:16:04.532733 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.532476    2578 scope.go:117] "RemoveContainer" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"
Jul 12 15:16:04.533052 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.533026    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"} err="failed to get container status \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": rpc error: code = NotFound desc = could not find container \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": container with ID starting with 3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f not found: ID does not exist"
Jul 12 15:16:04.533134 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.533056    2578 scope.go:117] "RemoveContainer" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"
Jul 12 15:16:04.533710 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.533691    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:16:04.533776 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.533724    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"} err="failed to get container status \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": rpc error: code = NotFound desc = could not find container \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": container with ID starting with 6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1 not found: ID does not exist"
Jul 12 15:16:04.533776 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.533750    2578 scope.go:117] "RemoveContainer" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"
Jul 12 15:16:04.534089 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.534065    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"} err="failed to get container status \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": rpc error: code = NotFound desc = could not find container \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": container with ID starting with 80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443 not found: ID does not exist"
Jul 12 15:16:04.534172 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.534089    2578 scope.go:117] "RemoveContainer" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"
Jul 12 15:16:04.534365 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.534339    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"} err="failed to get container status \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": rpc error: code = NotFound desc = could not find container \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": container with ID starting with f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231 not found: ID does not exist"
Jul 12 15:16:04.534451 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.534366    2578 scope.go:117] "RemoveContainer" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"
Jul 12 15:16:04.534638 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.534615    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"} err="failed to get container status \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": rpc error: code = NotFound desc = could not find container \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": container with ID starting with adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec not found: ID does not exist"
Jul 12 15:16:04.534698 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.534639    2578 scope.go:117] "RemoveContainer" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"
Jul 12 15:16:04.534919 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.534900    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"} err="failed to get container status \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": rpc error: code = NotFound desc = could not find container \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": container with ID starting with a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f not found: ID does not exist"
Jul 12 15:16:04.535013 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.534921    2578 scope.go:117] "RemoveContainer" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"
Jul 12 15:16:04.535176 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.535160    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"} err="failed to get container status \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": rpc error: code = NotFound desc = could not find container \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": container with ID starting with 73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b not found: ID does not exist"
Jul 12 15:16:04.535241 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.535178    2578 scope.go:117] "RemoveContainer" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"
Jul 12 15:16:04.535395 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.535373    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"} err="failed to get container status \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": rpc error: code = NotFound desc = could not find container \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": container with ID starting with 3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f not found: ID does not exist"
Jul 12 15:16:04.535437 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.535391    2578 scope.go:117] "RemoveContainer" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"
Jul 12 15:16:04.535610 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.535593    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"} err="failed to get container status \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": rpc error: code = NotFound desc = could not find container \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": container with ID starting with 6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1 not found: ID does not exist"
Jul 12 15:16:04.535679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.535611    2578 scope.go:117] "RemoveContainer" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"
Jul 12 15:16:04.535841 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.535819    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"} err="failed to get container status \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": rpc error: code = NotFound desc = could not find container \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": container with ID starting with 80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443 not found: ID does not exist"
Jul 12 15:16:04.535841 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.535838    2578 scope.go:117] "RemoveContainer" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"
Jul 12 15:16:04.536044 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536028    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"} err="failed to get container status \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": rpc error: code = NotFound desc = could not find container \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": container with ID starting with f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231 not found: ID does not exist"
Jul 12 15:16:04.536044 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536042    2578 scope.go:117] "RemoveContainer" containerID="adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"
Jul 12 15:16:04.536252 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536237    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec"} err="failed to get container status \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": rpc error: code = NotFound desc = could not find container \"adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec\": container with ID starting with adfcccae0510de5a1a5fd2553a1cb297f23ab078c907bd548622575b859c05ec not found: ID does not exist"
Jul 12 15:16:04.536252 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536251    2578 scope.go:117] "RemoveContainer" containerID="a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"
Jul 12 15:16:04.536440 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536422    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f"} err="failed to get container status \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": rpc error: code = NotFound desc = could not find container \"a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f\": container with ID starting with a92bab6f25bc4bda2b12ca6f196236742913117ede8c6feaa458fb40d484586f not found: ID does not exist"
Jul 12 15:16:04.536509 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536440    2578 scope.go:117] "RemoveContainer" containerID="73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"
Jul 12 15:16:04.536671 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536651    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b"} err="failed to get container status \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": rpc error: code = NotFound desc = could not find container \"73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b\": container with ID starting with 73c8156e60ac6fe0f56224db7b6bcd5d9358801a14a1f90c5dfacc33615a227b not found: ID does not exist"
Jul 12 15:16:04.536671 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536670    2578 scope.go:117] "RemoveContainer" containerID="3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"
Jul 12 15:16:04.536878 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536867    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f"} err="failed to get container status \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": rpc error: code = NotFound desc = could not find container \"3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f\": container with ID starting with 3d84457c40a83713230f42aebfb6dec90af0c8be4bff7bd90ad720d014b7ea5f not found: ID does not exist"
Jul 12 15:16:04.536943 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.536880    2578 scope.go:117] "RemoveContainer" containerID="6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"
Jul 12 15:16:04.537076 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.537061    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1"} err="failed to get container status \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": rpc error: code = NotFound desc = could not find container \"6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1\": container with ID starting with 6fba056f7b5c169f6d8f679bedafaa91d660968b5c31f5088db81f12828629d1 not found: ID does not exist"
Jul 12 15:16:04.537076 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.537076    2578 scope.go:117] "RemoveContainer" containerID="80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"
Jul 12 15:16:04.537241 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.537226    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443"} err="failed to get container status \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": rpc error: code = NotFound desc = could not find container \"80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443\": container with ID starting with 80c2a579f4edb517b7a64fb7449d6fb81f41df5e0ff8b98b7e8b4e5a8c4f1443 not found: ID does not exist"
Jul 12 15:16:04.537241 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.537240    2578 scope.go:117] "RemoveContainer" containerID="f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"
Jul 12 15:16:04.537435 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.537419    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231"} err="failed to get container status \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": rpc error: code = NotFound desc = could not find container \"f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231\": container with ID starting with f7f59e7d3bcb88ff2a66d79f4e69b4f1514495e531da6df0a602dcd20860a231 not found: ID does not exist"
Jul 12 15:16:04.543924 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.543900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.543999 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.543935    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q5g8\" (UniqueName: \"kubernetes.io/projected/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-kube-api-access-4q5g8\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.544054 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.543996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.544365 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544336    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-config\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544555    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544698    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544876    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-config-out\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545086 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.544905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545523 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.545172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545523 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.545203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545523 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.545230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-web-config\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545523 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.545292    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.545523 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.545429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.646780 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.646744    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-config\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.646956 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.646794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.646956 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.646835    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.646956 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.646863    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.646956 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.646891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.646956 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.646924    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.646956 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.646948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.646975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-config-out\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647126    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647142    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-web-config\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647235 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647221    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647742 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q5g8\" (UniqueName: \"kubernetes.io/projected/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-kube-api-access-4q5g8\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.647742 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.647281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.648999 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.648375    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.648999 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.648426    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.648999 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.648581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.649411 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.649386    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.650570 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.650002    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.650570 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.650125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.650570 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.650512    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.650895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.650870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-web-config\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.652406 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.652361    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.652497 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.652430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-config\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.652567 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.652530    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-config-out\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.653123 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.652884    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.653123 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.653052    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.653123 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.653092    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.653298 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.653209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.653466 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.653418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.653466 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.653447    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.653926 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.653883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.654324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.654300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.656298 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.656280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q5g8\" (UniqueName: \"kubernetes.io/projected/1eee6dc8-f4d6-4e95-9ee2-635cdbb45528-kube-api-access-4q5g8\") pod \"prometheus-k8s-0\" (UID: \"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.836148 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.836099    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:16:04.982837 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:04.982800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 15:16:04.985015 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:16:04.984988    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eee6dc8_f4d6_4e95_9ee2_635cdbb45528.slice/crio-357549563e720e67ecb0e6d290c94f5468c67d32c7898d3d49a7cde239c2fd6b WatchSource:0}: Error finding container 357549563e720e67ecb0e6d290c94f5468c67d32c7898d3d49a7cde239c2fd6b: Status 404 returned error can't find the container with id 357549563e720e67ecb0e6d290c94f5468c67d32c7898d3d49a7cde239c2fd6b
Jul 12 15:16:05.465443 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:05.465404    2578 generic.go:356] "Generic (PLEG): container finished" podID="1eee6dc8-f4d6-4e95-9ee2-635cdbb45528" containerID="0dfee3e6cb0c6bdd8a7400a34bc03b3f7c543e41d3eed6d0b007b5889ecfd88d" exitCode=0
Jul 12 15:16:05.465879 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:05.465476    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528","Type":"ContainerDied","Data":"0dfee3e6cb0c6bdd8a7400a34bc03b3f7c543e41d3eed6d0b007b5889ecfd88d"}
Jul 12 15:16:05.465879 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:05.465556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528","Type":"ContainerStarted","Data":"357549563e720e67ecb0e6d290c94f5468c67d32c7898d3d49a7cde239c2fd6b"}
Jul 12 15:16:06.155370 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.155340    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc4e00ac-07b0-4f01-a3ad-d36c7281e511" path="/var/lib/kubelet/pods/cc4e00ac-07b0-4f01-a3ad-d36c7281e511/volumes"
Jul 12 15:16:06.472222 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.472171    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8" event={"ID":"e705a0a9-f163-41de-8976-b9958cda7a89","Type":"ContainerStarted","Data":"2bd1ef68deec44a7f593bc9342d0afce987b0ce4ce99d6c7d74ac8f24e46a509"}
Jul 12 15:16:06.472655 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.472275    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8" event={"ID":"e705a0a9-f163-41de-8976-b9958cda7a89","Type":"ContainerStarted","Data":"2bd1fb50b8da7e7e6e78fcf86f931a71f9749e7c97f454956427882ddd523c82"}
Jul 12 15:16:06.472655 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.472296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8" event={"ID":"e705a0a9-f163-41de-8976-b9958cda7a89","Type":"ContainerStarted","Data":"21d3a1127fe33c27d580a1f59813eeee15a7c303bf164c40df9db257934298dc"}
Jul 12 15:16:06.475519 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.475485    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528","Type":"ContainerStarted","Data":"2a1406edb772d006ba1a6aacca4a6e6c1a01bce84bfadacef4c410cf6c73ec7b"}
Jul 12 15:16:06.475633 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.475554    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528","Type":"ContainerStarted","Data":"1571403e959bc6b8b27571d0ca720c18c08609aaf60c748509e62a45bc4e152e"}
Jul 12 15:16:06.475633 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.475599    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528","Type":"ContainerStarted","Data":"e30c27a6387df32b8f751b18189c350481e05c2cdc3dd91e6a2b1ffe6bd8aa61"}
Jul 12 15:16:06.475760 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.475637    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528","Type":"ContainerStarted","Data":"144114f800c1b6f93866b7885c288d8cb4af886fe9cdf2e39e653a7954fe8eaa"}
Jul 12 15:16:06.475760 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.475678    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528","Type":"ContainerStarted","Data":"7860a8fe0996da0732222ab91f1995170acb69ada6c5945efa1660e41659c794"}
Jul 12 15:16:06.475760 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.475717    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1eee6dc8-f4d6-4e95-9ee2-635cdbb45528","Type":"ContainerStarted","Data":"12d50fa3948dba8d05774be339002f28d3bc103b296434a491f250dd5a1b0f27"}
Jul 12 15:16:06.494300 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.494252    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-b6f595dcc-mqrs8" podStartSLOduration=1.410949544 podStartE2EDuration="3.494236623s" totalImagesPullingTime="2.083287079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:16:06.491551026 +0000 UTC m=+384.859151871" watchObservedRunningTime="2026-07-12 15:16:06.494236623 +0000 UTC m=+384.861837476"
Jul 12 15:16:06.518829 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:06.518762    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.5014300240000003 podStartE2EDuration="2.518748393s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.318369ms" isStatefulPod=false podCreationTimestamp="2026-07-12 15:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:16:06.516226704 +0000 UTC m=+384.883827563" watchObservedRunningTime="2026-07-12 15:16:06.518748393 +0000 UTC m=+384.886349262"
Jul 12 15:16:09.837036 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:16:09.836998    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:17:04.836748 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:17:04.836716    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:17:04.851950 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:17:04.851927    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:17:05.721409 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:17:05.721384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 15:19:39.083787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.083752    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"]
Jul 12 15:19:39.087192 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.087172    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.090341 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.090319    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 12 15:19:39.090463 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.090355    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 12 15:19:39.090463 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.090361    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 12 15:19:39.091359 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.091342    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9xpzx\""
Jul 12 15:19:39.091454 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.091344    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 12 15:19:39.116525 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.116499    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"]
Jul 12 15:19:39.159246 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.159210    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"]
Jul 12 15:19:39.164132 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.164104    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.166996 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.166974    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 12 15:19:39.167684 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.167661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 12 15:19:39.167947 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.167912    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zxnl6\""
Jul 12 15:19:39.168144 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.168120    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 12 15:19:39.168261 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.168232    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 12 15:19:39.168261 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.168234    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 12 15:19:39.171582 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.171562    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.171696 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.171619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xztbt\" (UniqueName: \"kubernetes.io/projected/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-kube-api-access-xztbt\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.171696 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.171656    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-tmp\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.171696 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.171687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-manifests\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.171879 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.171744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-webhook-cert\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.175743 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.175724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"]
Jul 12 15:19:39.273116 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273084    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wvb9\" (UniqueName: \"kubernetes.io/projected/e501d677-e1e9-445a-a492-c0096e89a1d8-kube-api-access-2wvb9\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.273324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273127    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e501d677-e1e9-445a-a492-c0096e89a1d8-cert\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.273324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273154    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-webhook-cert\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.273324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.273492 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e501d677-e1e9-445a-a492-c0096e89a1d8-manager-config\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.273492 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xztbt\" (UniqueName: \"kubernetes.io/projected/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-kube-api-access-xztbt\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.273492 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e501d677-e1e9-445a-a492-c0096e89a1d8-metrics-cert\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.273636 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-tmp\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.273636 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.273558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-manifests\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.274047 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.274021    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-manifests\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.275575 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.275556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-tmp\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.276163 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.276142    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.276246 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.276163    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-webhook-cert\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.283223 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.283201    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xztbt\" (UniqueName: \"kubernetes.io/projected/9dc0240a-d3d7-493c-93fe-89a5cb0b1216-kube-api-access-xztbt\") pod \"opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc\" (UID: \"9dc0240a-d3d7-493c-93fe-89a5cb0b1216\") " pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.374515 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.374426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e501d677-e1e9-445a-a492-c0096e89a1d8-manager-config\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.374515 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.374472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e501d677-e1e9-445a-a492-c0096e89a1d8-metrics-cert\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.374708 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.374593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wvb9\" (UniqueName: \"kubernetes.io/projected/e501d677-e1e9-445a-a492-c0096e89a1d8-kube-api-access-2wvb9\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.374708 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.374653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e501d677-e1e9-445a-a492-c0096e89a1d8-cert\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.375220 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.375193    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e501d677-e1e9-445a-a492-c0096e89a1d8-manager-config\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.376909 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.376886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e501d677-e1e9-445a-a492-c0096e89a1d8-cert\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.376997 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.376933    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e501d677-e1e9-445a-a492-c0096e89a1d8-metrics-cert\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.389589 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.389560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wvb9\" (UniqueName: \"kubernetes.io/projected/e501d677-e1e9-445a-a492-c0096e89a1d8-kube-api-access-2wvb9\") pod \"lws-controller-manager-676dd59b87-6nqkk\" (UID: \"e501d677-e1e9-445a-a492-c0096e89a1d8\") " pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.397453 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.397434    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:39.473351 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.473322    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:39.535499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.534157    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"]
Jul 12 15:19:39.538395 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:19:39.538372    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc0240a_d3d7_493c_93fe_89a5cb0b1216.slice/crio-4464db0036256084c8b319860001a04b5b073f9e65b4d0feb50c4700e333ff9f WatchSource:0}: Error finding container 4464db0036256084c8b319860001a04b5b073f9e65b4d0feb50c4700e333ff9f: Status 404 returned error can't find the container with id 4464db0036256084c8b319860001a04b5b073f9e65b4d0feb50c4700e333ff9f
Jul 12 15:19:39.621729 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:39.621696    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"]
Jul 12 15:19:39.625450 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:19:39.625390    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode501d677_e1e9_445a_a492_c0096e89a1d8.slice/crio-c329a6c8943b65362ceb44406c6afd90e609b4c0a89fa0cc014cb61ca637d4b4 WatchSource:0}: Error finding container c329a6c8943b65362ceb44406c6afd90e609b4c0a89fa0cc014cb61ca637d4b4: Status 404 returned error can't find the container with id c329a6c8943b65362ceb44406c6afd90e609b4c0a89fa0cc014cb61ca637d4b4
Jul 12 15:19:40.300347 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:40.300265    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk" event={"ID":"e501d677-e1e9-445a-a492-c0096e89a1d8","Type":"ContainerStarted","Data":"c329a6c8943b65362ceb44406c6afd90e609b4c0a89fa0cc014cb61ca637d4b4"}
Jul 12 15:19:40.302733 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:40.302701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc" event={"ID":"9dc0240a-d3d7-493c-93fe-89a5cb0b1216","Type":"ContainerStarted","Data":"4464db0036256084c8b319860001a04b5b073f9e65b4d0feb50c4700e333ff9f"}
Jul 12 15:19:44.323590 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:44.323548    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc" event={"ID":"9dc0240a-d3d7-493c-93fe-89a5cb0b1216","Type":"ContainerStarted","Data":"040be0daabc33ad83c716ab55af007efacaef3da4adead9261073753055355af"}
Jul 12 15:19:44.324922 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:44.324897    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk" event={"ID":"e501d677-e1e9-445a-a492-c0096e89a1d8","Type":"ContainerStarted","Data":"5842df16e645019956d32477251a2c90f2052767a5d1a0ce48752e3a7f20299e"}
Jul 12 15:19:44.325057 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:44.325037    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:44.366879 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:44.366832    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk" podStartSLOduration=1.263466429 podStartE2EDuration="5.366816325s" totalImagesPullingTime="4.103349896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:19:44.365057213 +0000 UTC m=+602.732658061" watchObservedRunningTime="2026-07-12 15:19:44.366816325 +0000 UTC m=+602.734417236"
Jul 12 15:19:45.330384 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:45.330304    2578 generic.go:356] "Generic (PLEG): container finished" podID="9dc0240a-d3d7-493c-93fe-89a5cb0b1216" containerID="040be0daabc33ad83c716ab55af007efacaef3da4adead9261073753055355af" exitCode=0
Jul 12 15:19:45.330764 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:45.330444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc" event={"ID":"9dc0240a-d3d7-493c-93fe-89a5cb0b1216","Type":"ContainerDied","Data":"040be0daabc33ad83c716ab55af007efacaef3da4adead9261073753055355af"}
Jul 12 15:19:45.330901 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:45.330885    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 15:19:46.336486 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:46.336453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc" event={"ID":"9dc0240a-d3d7-493c-93fe-89a5cb0b1216","Type":"ContainerStarted","Data":"4770625a8c048ac0a48d7281d8fc4baee93356d7ce70bee777eaa1d3168ab234"}
Jul 12 15:19:46.336918 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:46.336574    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:19:46.357058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:46.357014    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc" podStartSLOduration=1.890390443 podStartE2EDuration="7.356996999s" totalImagesPullingTime="4.43399715s" totalInitContainerRuntime="1.032609406s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:19:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:19:46.355771694 +0000 UTC m=+604.723372541" watchObservedRunningTime="2026-07-12 15:19:46.356996999 +0000 UTC m=+604.724597845"
Jul 12 15:19:55.332484 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:55.332451    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-676dd59b87-6nqkk"
Jul 12 15:19:57.343723 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:19:57.343692    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc"
Jul 12 15:20:04.919324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:04.919283    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"]
Jul 12 15:20:04.937506 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:04.937474    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"]
Jul 12 15:20:04.937657 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:04.937596    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:04.940554 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:04.940532    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 12 15:20:04.940797 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:04.940781    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 12 15:20:04.940972 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:04.940948    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 12 15:20:04.941442 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:04.941425    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hhmtr\""
Jul 12 15:20:04.941652 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:04.941637    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 12 15:20:05.109096 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.109065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59270970-f6ab-4790-ba2d-a6a7ec2ec646-tls-certs\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.109266 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.109117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59270970-f6ab-4790-ba2d-a6a7ec2ec646-tmp\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.109266 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.109146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5wch\" (UniqueName: \"kubernetes.io/projected/59270970-f6ab-4790-ba2d-a6a7ec2ec646-kube-api-access-x5wch\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.209978 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.209892    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59270970-f6ab-4790-ba2d-a6a7ec2ec646-tls-certs\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.209978 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.209948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59270970-f6ab-4790-ba2d-a6a7ec2ec646-tmp\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.209978 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.209978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wch\" (UniqueName: \"kubernetes.io/projected/59270970-f6ab-4790-ba2d-a6a7ec2ec646-kube-api-access-x5wch\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.212206 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.212145    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59270970-f6ab-4790-ba2d-a6a7ec2ec646-tmp\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.212399 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.212380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59270970-f6ab-4790-ba2d-a6a7ec2ec646-tls-certs\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.217700 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.217674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5wch\" (UniqueName: \"kubernetes.io/projected/59270970-f6ab-4790-ba2d-a6a7ec2ec646-kube-api-access-x5wch\") pod \"kube-auth-proxy-6fd5b5f7f7-2dmkl\" (UID: \"59270970-f6ab-4790-ba2d-a6a7ec2ec646\") " pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.247753 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.247725    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"
Jul 12 15:20:05.375226 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.375203    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl"]
Jul 12 15:20:05.377628 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:20:05.377598    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59270970_f6ab_4790_ba2d_a6a7ec2ec646.slice/crio-6b0e8c413cedbd4badd8f912106d987cdad0938e4e7f911154d14115fb9b1b99 WatchSource:0}: Error finding container 6b0e8c413cedbd4badd8f912106d987cdad0938e4e7f911154d14115fb9b1b99: Status 404 returned error can't find the container with id 6b0e8c413cedbd4badd8f912106d987cdad0938e4e7f911154d14115fb9b1b99
Jul 12 15:20:05.421622 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:05.421590    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl" event={"ID":"59270970-f6ab-4790-ba2d-a6a7ec2ec646","Type":"ContainerStarted","Data":"6b0e8c413cedbd4badd8f912106d987cdad0938e4e7f911154d14115fb9b1b99"}
Jul 12 15:20:08.443398 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:08.443301    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl" event={"ID":"59270970-f6ab-4790-ba2d-a6a7ec2ec646","Type":"ContainerStarted","Data":"9c1a27a41df4a917e9751294d6c92dacd5be5a2497ebba40f73a1494541a630e"}
Jul 12 15:20:08.459039 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:20:08.458990    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6fd5b5f7f7-2dmkl" podStartSLOduration=1.6466790329999998 podStartE2EDuration="4.458973511s" totalImagesPullingTime="2.812294478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:08.458301728 +0000 UTC m=+626.825902571" watchObservedRunningTime="2026-07-12 15:20:08.458973511 +0000 UTC m=+626.826574357"
Jul 12 15:21:17.128216 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.128161    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-655b7dc8c7-4hkcc"]
Jul 12 15:21:17.131414 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.131398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.134084 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.134054    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 12 15:21:17.134212 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.134091    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wwtcp\""
Jul 12 15:21:17.135550 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.135534    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 12 15:21:17.135655 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.135565    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 12 15:21:17.135655 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.135638    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 12 15:21:17.135776 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.135650    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 12 15:21:17.135776 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.135651    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 12 15:21:17.135776 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.135766    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 12 15:21:17.140561 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.140536    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 12 15:21:17.142997 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.142975    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-655b7dc8c7-4hkcc"]
Jul 12 15:21:17.226149 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.226121    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-oauth-config\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.226311 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.226173    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7cv2\" (UniqueName: \"kubernetes.io/projected/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-kube-api-access-z7cv2\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.226311 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.226196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-config\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.226311 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.226236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-service-ca\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.226311 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.226253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-oauth-serving-cert\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.226488 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.226335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-serving-cert\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.226488 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.226376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-trusted-ca-bundle\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.327325 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.327289    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-oauth-config\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.327514 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.327364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7cv2\" (UniqueName: \"kubernetes.io/projected/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-kube-api-access-z7cv2\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.327514 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.327389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-config\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.327514 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.327435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-service-ca\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.327676 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.327619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-oauth-serving-cert\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.327748 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.327709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-serving-cert\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.327748 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.327736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-trusted-ca-bundle\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.328330 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.328306    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-service-ca\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.328502 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.328356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-oauth-serving-cert\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.328502 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.328372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-config\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.328663 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.328547    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-trusted-ca-bundle\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.329895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.329873    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-oauth-config\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.330227 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.330200    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-console-serving-cert\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.334865 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.334845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7cv2\" (UniqueName: \"kubernetes.io/projected/2c91dfc1-f39a-46d9-ba0b-345ba0a9863b-kube-api-access-z7cv2\") pod \"console-655b7dc8c7-4hkcc\" (UID: \"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b\") " pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.442341 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.442254    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:17.567646 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.567616    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-655b7dc8c7-4hkcc"]
Jul 12 15:21:17.570846 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:21:17.570817    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c91dfc1_f39a_46d9_ba0b_345ba0a9863b.slice/crio-d71691f78a9a5bb1e5c846cbd697473d2ba285a6e232b10d8d6d75b9a127257a WatchSource:0}: Error finding container d71691f78a9a5bb1e5c846cbd697473d2ba285a6e232b10d8d6d75b9a127257a: Status 404 returned error can't find the container with id d71691f78a9a5bb1e5c846cbd697473d2ba285a6e232b10d8d6d75b9a127257a
Jul 12 15:21:17.737700 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.737612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-655b7dc8c7-4hkcc" event={"ID":"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b","Type":"ContainerStarted","Data":"7ba2fce34f5f90b59d55483aec83cf5db00499f6efe5b34e2165f6632a6e9a02"}
Jul 12 15:21:17.737700 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.737665    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-655b7dc8c7-4hkcc" event={"ID":"2c91dfc1-f39a-46d9-ba0b-345ba0a9863b","Type":"ContainerStarted","Data":"d71691f78a9a5bb1e5c846cbd697473d2ba285a6e232b10d8d6d75b9a127257a"}
Jul 12 15:21:17.754833 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:17.754771    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-655b7dc8c7-4hkcc" podStartSLOduration=0.754755852 podStartE2EDuration="754.755852ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:17.753501512 +0000 UTC m=+696.121102369" watchObservedRunningTime="2026-07-12 15:21:17.754755852 +0000 UTC m=+696.122356698"
Jul 12 15:21:27.442957 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:27.442901    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:27.442957 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:27.442976    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:27.447702 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:27.447677    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:27.788237 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:27.788144    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-655b7dc8c7-4hkcc"
Jul 12 15:21:53.405007 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.404971    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"]
Jul 12 15:21:53.408078 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.408060    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.410679 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.410655    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 12 15:21:53.410838 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.410707    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 12 15:21:53.410838 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.410719    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-l2jxw\""
Jul 12 15:21:53.410838 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.410719    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 12 15:21:53.411915 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.411898    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 12 15:21:53.414650 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.414628    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"]
Jul 12 15:21:53.440345 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.440320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1780c8d-7a72-49a4-9d6c-03014b656975-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.440468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.440351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1780c8d-7a72-49a4-9d6c-03014b656975-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.440468 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.440431    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgkfx\" (UniqueName: \"kubernetes.io/projected/e1780c8d-7a72-49a4-9d6c-03014b656975-kube-api-access-pgkfx\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.541590 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.541559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1780c8d-7a72-49a4-9d6c-03014b656975-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.541590 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.541594    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1780c8d-7a72-49a4-9d6c-03014b656975-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.541851 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.541640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgkfx\" (UniqueName: \"kubernetes.io/projected/e1780c8d-7a72-49a4-9d6c-03014b656975-kube-api-access-pgkfx\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.542336 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.542312    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1780c8d-7a72-49a4-9d6c-03014b656975-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.543962 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.543943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1780c8d-7a72-49a4-9d6c-03014b656975-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.553077 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.553056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgkfx\" (UniqueName: \"kubernetes.io/projected/e1780c8d-7a72-49a4-9d6c-03014b656975-kube-api-access-pgkfx\") pod \"kuadrant-console-plugin-66b5774985-d4d44\" (UID: \"e1780c8d-7a72-49a4-9d6c-03014b656975\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.728316 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.728234    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"
Jul 12 15:21:53.883630 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.883598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44"]
Jul 12 15:21:53.886922 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:21:53.886892    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1780c8d_7a72_49a4_9d6c_03014b656975.slice/crio-6ce349c1258fb140955e51a29b346ac0a05e3f46ea6e14a75d0841dab15db6fb WatchSource:0}: Error finding container 6ce349c1258fb140955e51a29b346ac0a05e3f46ea6e14a75d0841dab15db6fb: Status 404 returned error can't find the container with id 6ce349c1258fb140955e51a29b346ac0a05e3f46ea6e14a75d0841dab15db6fb
Jul 12 15:21:53.894133 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:21:53.894108    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44" event={"ID":"e1780c8d-7a72-49a4-9d6c-03014b656975","Type":"ContainerStarted","Data":"6ce349c1258fb140955e51a29b346ac0a05e3f46ea6e14a75d0841dab15db6fb"}
Jul 12 15:22:03.647510 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.647469    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"]
Jul 12 15:22:03.679900 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.679831    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"]
Jul 12 15:22:03.680062 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.679963    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:03.682723 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.682696    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kf5n9\""
Jul 12 15:22:03.739481 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.739440    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6wvv\" (UniqueName: \"kubernetes.io/projected/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-kube-api-access-d6wvv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" (UID: \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:03.739662 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.739566    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" (UID: \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:03.840827 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.840772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6wvv\" (UniqueName: \"kubernetes.io/projected/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-kube-api-access-d6wvv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" (UID: \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:03.841019 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.840914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" (UID: \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:03.841325 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.841290    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" (UID: \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:03.857324 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.857294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6wvv\" (UniqueName: \"kubernetes.io/projected/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-kube-api-access-d6wvv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" (UID: \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:03.993394 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:03.993073    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:04.231414 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.231378    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"]
Jul 12 15:22:04.236731 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.236699    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"]
Jul 12 15:22:04.295351 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.295275    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"]
Jul 12 15:22:04.310412 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.310381    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"
Jul 12 15:22:04.311795 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.311762    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"]
Jul 12 15:22:04.313971 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.313937    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-crdgg\""
Jul 12 15:22:04.446099 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.446059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfcs2\" (UniqueName: \"kubernetes.io/projected/7161441b-fa0e-42fd-af31-97ade7f92adf-kube-api-access-lfcs2\") pod \"limitador-operator-controller-manager-859d7b78bd-h824m\" (UID: \"7161441b-fa0e-42fd-af31-97ade7f92adf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"
Jul 12 15:22:04.546664 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.546583    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfcs2\" (UniqueName: \"kubernetes.io/projected/7161441b-fa0e-42fd-af31-97ade7f92adf-kube-api-access-lfcs2\") pod \"limitador-operator-controller-manager-859d7b78bd-h824m\" (UID: \"7161441b-fa0e-42fd-af31-97ade7f92adf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"
Jul 12 15:22:04.559048 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.559017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfcs2\" (UniqueName: \"kubernetes.io/projected/7161441b-fa0e-42fd-af31-97ade7f92adf-kube-api-access-lfcs2\") pod \"limitador-operator-controller-manager-859d7b78bd-h824m\" (UID: \"7161441b-fa0e-42fd-af31-97ade7f92adf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"
Jul 12 15:22:04.626122 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:04.626082    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"
Jul 12 15:22:15.028587 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:15.028511    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"]
Jul 12 15:22:15.031126 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:22:15.031100    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7161441b_fa0e_42fd_af31_97ade7f92adf.slice/crio-f1903804b9acfcad747274807813da50848219c4ecd5fe045fbde81e28090596 WatchSource:0}: Error finding container f1903804b9acfcad747274807813da50848219c4ecd5fe045fbde81e28090596: Status 404 returned error can't find the container with id f1903804b9acfcad747274807813da50848219c4ecd5fe045fbde81e28090596
Jul 12 15:22:16.014480 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:16.014449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m" event={"ID":"7161441b-fa0e-42fd-af31-97ade7f92adf","Type":"ContainerStarted","Data":"f1903804b9acfcad747274807813da50848219c4ecd5fe045fbde81e28090596"}
Jul 12 15:22:16.016027 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:16.016001    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44" event={"ID":"e1780c8d-7a72-49a4-9d6c-03014b656975","Type":"ContainerStarted","Data":"616da143a12b3914a751c92fb1c3bcb81379cc80b0e16c19da3baeb123f00670"}
Jul 12 15:22:16.032677 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:16.032624    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d4d44" podStartSLOduration=1.947039013 podStartE2EDuration="23.032604763s" totalImagesPullingTime="21.08556575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:22:16.032303052 +0000 UTC m=+754.399903900" watchObservedRunningTime="2026-07-12 15:22:16.032604763 +0000 UTC m=+754.400205611"
Jul 12 15:22:17.810036 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:22:17.809993    2578 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-2ng74_kuadrant-system_08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a_0(af3eec3631b1760d8942d00e57a64d8e3231cddde3984ed47f331b4a9faad9b9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-2ng74 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 12 15:22:17.810309 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:22:17.810062    2578 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-2ng74_kuadrant-system_08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a_0(af3eec3631b1760d8942d00e57a64d8e3231cddde3984ed47f331b4a9faad9b9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-2ng74 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:18.024563 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:18.024533    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:18.026842 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:22:18.026796    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" is forbidden: User \"system:node:ip-10-0-142-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-238.ec2.internal' and this object" podUID="08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:18.057442 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:18.057412    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:18.059723 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:22:18.059696    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" is forbidden: User \"system:node:ip-10-0-142-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-238.ec2.internal' and this object" podUID="08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:18.181321 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:18.181291    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6wvv\" (UniqueName: \"kubernetes.io/projected/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-kube-api-access-d6wvv\") pod \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\" (UID: \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\") "
Jul 12 15:22:18.181485 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:18.181394    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-extensions-socket-volume\") pod \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\" (UID: \"08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a\") "
Jul 12 15:22:18.181689 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:18.181665    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a" (UID: "08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:22:18.183306 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:18.183285    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-kube-api-access-d6wvv" (OuterVolumeSpecName: "kube-api-access-d6wvv") pod "08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a" (UID: "08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a"). InnerVolumeSpecName "kube-api-access-d6wvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:22:18.282266 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:18.282233    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6wvv\" (UniqueName: \"kubernetes.io/projected/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-kube-api-access-d6wvv\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:22:18.282266 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:18.282259    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a-extensions-socket-volume\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:22:19.030680 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:19.030626    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m" event={"ID":"7161441b-fa0e-42fd-af31-97ade7f92adf","Type":"ContainerStarted","Data":"9a094410bad4931c91c8c7ad80f4a27661b7e235540e9664ec3054bfcdbcd9f4"}
Jul 12 15:22:19.031059 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:19.030833    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"
Jul 12 15:22:19.031059 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:19.030882    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:19.047242 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:19.047197    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m" podStartSLOduration=11.870475428 podStartE2EDuration="15.047182248s" totalImagesPullingTime="3.17670682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:22:19.045884801 +0000 UTC m=+757.413485649" watchObservedRunningTime="2026-07-12 15:22:19.047182248 +0000 UTC m=+757.414783171"
Jul 12 15:22:19.047963 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:22:19.047934    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2ng74\" is forbidden: User \"system:node:ip-10-0-142-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-238.ec2.internal' and this object" podUID="08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2ng74"
Jul 12 15:22:20.154743 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:20.154713    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a" path="/var/lib/kubelet/pods/08a37cb9-34f6-4fb8-9f8f-d603d55e1e8a/volumes"
Jul 12 15:22:30.038120 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:30.038087    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h824m"
Jul 12 15:22:48.201993 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.201949    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-g97zv"]
Jul 12 15:22:48.226435 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.226398    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-g97zv"]
Jul 12 15:22:48.226600 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.226502    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g97zv"
Jul 12 15:22:48.229037 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.229013    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ckn5t\""
Jul 12 15:22:48.348411 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.348371    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhb57\" (UniqueName: \"kubernetes.io/projected/7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c-kube-api-access-hhb57\") pod \"authorino-58cd955675-g97zv\" (UID: \"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c\") " pod="kuadrant-system/authorino-58cd955675-g97zv"
Jul 12 15:22:48.398492 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.398452    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-2k6r2"]
Jul 12 15:22:48.401972 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.401950    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2k6r2"
Jul 12 15:22:48.407531 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.407502    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2k6r2"]
Jul 12 15:22:48.449784 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.449740    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dvkg\" (UniqueName: \"kubernetes.io/projected/58e82fb4-e6b3-4675-804c-9e78b7760bce-kube-api-access-8dvkg\") pod \"authorino-75d556bd77-2k6r2\" (UID: \"58e82fb4-e6b3-4675-804c-9e78b7760bce\") " pod="kuadrant-system/authorino-75d556bd77-2k6r2"
Jul 12 15:22:48.449983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.449851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhb57\" (UniqueName: \"kubernetes.io/projected/7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c-kube-api-access-hhb57\") pod \"authorino-58cd955675-g97zv\" (UID: \"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c\") " pod="kuadrant-system/authorino-58cd955675-g97zv"
Jul 12 15:22:48.457365 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.457294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhb57\" (UniqueName: \"kubernetes.io/projected/7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c-kube-api-access-hhb57\") pod \"authorino-58cd955675-g97zv\" (UID: \"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c\") " pod="kuadrant-system/authorino-58cd955675-g97zv"
Jul 12 15:22:48.536262 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.536224    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g97zv"
Jul 12 15:22:48.550766 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.550734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dvkg\" (UniqueName: \"kubernetes.io/projected/58e82fb4-e6b3-4675-804c-9e78b7760bce-kube-api-access-8dvkg\") pod \"authorino-75d556bd77-2k6r2\" (UID: \"58e82fb4-e6b3-4675-804c-9e78b7760bce\") " pod="kuadrant-system/authorino-75d556bd77-2k6r2"
Jul 12 15:22:48.559162 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.559125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dvkg\" (UniqueName: \"kubernetes.io/projected/58e82fb4-e6b3-4675-804c-9e78b7760bce-kube-api-access-8dvkg\") pod \"authorino-75d556bd77-2k6r2\" (UID: \"58e82fb4-e6b3-4675-804c-9e78b7760bce\") " pod="kuadrant-system/authorino-75d556bd77-2k6r2"
Jul 12 15:22:48.712189 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.712084    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2k6r2"
Jul 12 15:22:48.718993 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.718961    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-g97zv"]
Jul 12 15:22:48.725280 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:22:48.725252    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7cf5deca_f55e_45c8_af0e_9c16ec6e2b9c.slice/crio-f6a3b96d7326544bab9397a74be4e1b19cc62bd5e00a71e3c65fb16972accb3e WatchSource:0}: Error finding container f6a3b96d7326544bab9397a74be4e1b19cc62bd5e00a71e3c65fb16972accb3e: Status 404 returned error can't find the container with id f6a3b96d7326544bab9397a74be4e1b19cc62bd5e00a71e3c65fb16972accb3e
Jul 12 15:22:48.850610 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:48.850575    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2k6r2"]
Jul 12 15:22:48.852582 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:22:48.852553    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58e82fb4_e6b3_4675_804c_9e78b7760bce.slice/crio-a7a8cdb2bc2ef799a438a8dd3b681572e415c1cf223217059be81f083dd329d3 WatchSource:0}: Error finding container a7a8cdb2bc2ef799a438a8dd3b681572e415c1cf223217059be81f083dd329d3: Status 404 returned error can't find the container with id a7a8cdb2bc2ef799a438a8dd3b681572e415c1cf223217059be81f083dd329d3
Jul 12 15:22:49.174055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:49.174014    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2k6r2" event={"ID":"58e82fb4-e6b3-4675-804c-9e78b7760bce","Type":"ContainerStarted","Data":"a7a8cdb2bc2ef799a438a8dd3b681572e415c1cf223217059be81f083dd329d3"}
Jul 12 15:22:49.175116 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:49.175088    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g97zv" event={"ID":"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c","Type":"ContainerStarted","Data":"f6a3b96d7326544bab9397a74be4e1b19cc62bd5e00a71e3c65fb16972accb3e"}
Jul 12 15:22:53.201968 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:53.201919    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2k6r2" event={"ID":"58e82fb4-e6b3-4675-804c-9e78b7760bce","Type":"ContainerStarted","Data":"a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b"}
Jul 12 15:22:53.203426 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:53.203404    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g97zv" event={"ID":"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c","Type":"ContainerStarted","Data":"42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14"}
Jul 12 15:22:53.215917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:53.215869    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-2k6r2" podStartSLOduration=1.8237675690000001 podStartE2EDuration="5.215851367s" totalImagesPullingTime="3.392083798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:22:53.215345681 +0000 UTC m=+791.582946530" watchObservedRunningTime="2026-07-12 15:22:53.215851367 +0000 UTC m=+791.583452192"
Jul 12 15:22:53.228876 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:53.228785    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-g97zv" podStartSLOduration=1.7220321090000001 podStartE2EDuration="5.228761591s" totalImagesPullingTime="3.506729482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:22:53.227949267 +0000 UTC m=+791.595550118" watchObservedRunningTime="2026-07-12 15:22:53.228761591 +0000 UTC m=+791.596362443"
Jul 12 15:22:53.246061 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:53.246026    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-g97zv"]
Jul 12 15:22:55.211977 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:55.211928    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-g97zv" podUID="7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c" containerName="authorino" containerID="cri-o://42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14" gracePeriod=30
Jul 12 15:22:55.456164 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:55.456128    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g97zv"
Jul 12 15:22:55.519080 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:55.518980    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhb57\" (UniqueName: \"kubernetes.io/projected/7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c-kube-api-access-hhb57\") pod \"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c\" (UID: \"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c\") "
Jul 12 15:22:55.521252 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:55.521228    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c-kube-api-access-hhb57" (OuterVolumeSpecName: "kube-api-access-hhb57") pod "7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c" (UID: "7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c"). InnerVolumeSpecName "kube-api-access-hhb57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:22:55.620172 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:55.620123    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhb57\" (UniqueName: \"kubernetes.io/projected/7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c-kube-api-access-hhb57\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:22:56.217227 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.217183    2578 generic.go:356] "Generic (PLEG): container finished" podID="7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c" containerID="42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14" exitCode=0
Jul 12 15:22:56.217637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.217238    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g97zv"
Jul 12 15:22:56.217637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.217263    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g97zv" event={"ID":"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c","Type":"ContainerDied","Data":"42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14"}
Jul 12 15:22:56.217637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.217306    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g97zv" event={"ID":"7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c","Type":"ContainerDied","Data":"f6a3b96d7326544bab9397a74be4e1b19cc62bd5e00a71e3c65fb16972accb3e"}
Jul 12 15:22:56.217637 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.217323    2578 scope.go:117] "RemoveContainer" containerID="42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14"
Jul 12 15:22:56.225665 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.225646    2578 scope.go:117] "RemoveContainer" containerID="42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14"
Jul 12 15:22:56.225974 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:22:56.225953    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14\": container with ID starting with 42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14 not found: ID does not exist" containerID="42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14"
Jul 12 15:22:56.226047 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.225987    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14"} err="failed to get container status \"42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14\": rpc error: code = NotFound desc = could not find container \"42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14\": container with ID starting with 42ff0d95256da0248d0da74b4a190ffa7f0c3c036f2985bb225e382937a25f14 not found: ID does not exist"
Jul 12 15:22:56.233650 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.233622    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-g97zv"]
Jul 12 15:22:56.236897 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:56.236874    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-g97zv"]
Jul 12 15:22:58.155041 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:22:58.155006    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c" path="/var/lib/kubelet/pods/7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c/volumes"
Jul 12 15:23:17.685091 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.685050    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-dd77f575d-ftchp"]
Jul 12 15:23:17.685511 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.685424    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c" containerName="authorino"
Jul 12 15:23:17.685511 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.685436    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c" containerName="authorino"
Jul 12 15:23:17.685511 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.685491    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf5deca-f55e-45c8-af0e-9c16ec6e2b9c" containerName="authorino"
Jul 12 15:23:17.688578 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.688560    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:23:17.691016 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.690990    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 12 15:23:17.694397 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.694363    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-dd77f575d-ftchp"]
Jul 12 15:23:17.821900 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.821843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psgrg\" (UniqueName: \"kubernetes.io/projected/9d6d9ade-f3c1-41a4-a488-993f8d236852-kube-api-access-psgrg\") pod \"authorino-dd77f575d-ftchp\" (UID: \"9d6d9ade-f3c1-41a4-a488-993f8d236852\") " pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:23:17.822085 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.821972    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d6d9ade-f3c1-41a4-a488-993f8d236852-tls-cert\") pod \"authorino-dd77f575d-ftchp\" (UID: \"9d6d9ade-f3c1-41a4-a488-993f8d236852\") " pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:23:17.923117 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.923078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psgrg\" (UniqueName: \"kubernetes.io/projected/9d6d9ade-f3c1-41a4-a488-993f8d236852-kube-api-access-psgrg\") pod \"authorino-dd77f575d-ftchp\" (UID: \"9d6d9ade-f3c1-41a4-a488-993f8d236852\") " pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:23:17.923237 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.923168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d6d9ade-f3c1-41a4-a488-993f8d236852-tls-cert\") pod \"authorino-dd77f575d-ftchp\" (UID: \"9d6d9ade-f3c1-41a4-a488-993f8d236852\") " pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:23:17.925619 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.925584    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d6d9ade-f3c1-41a4-a488-993f8d236852-tls-cert\") pod \"authorino-dd77f575d-ftchp\" (UID: \"9d6d9ade-f3c1-41a4-a488-993f8d236852\") " pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:23:17.930664 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.930642    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psgrg\" (UniqueName: \"kubernetes.io/projected/9d6d9ade-f3c1-41a4-a488-993f8d236852-kube-api-access-psgrg\") pod \"authorino-dd77f575d-ftchp\" (UID: \"9d6d9ade-f3c1-41a4-a488-993f8d236852\") " pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:23:17.998448 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:17.998351    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:23:18.130726 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:18.130701    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-dd77f575d-ftchp"]
Jul 12 15:23:18.133645 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:23:18.133620    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d6d9ade_f3c1_41a4_a488_993f8d236852.slice/crio-634e97b1fd8a840bfa7adf392f77199a6d328d1eabda679b469ebd828e522558 WatchSource:0}: Error finding container 634e97b1fd8a840bfa7adf392f77199a6d328d1eabda679b469ebd828e522558: Status 404 returned error can't find the container with id 634e97b1fd8a840bfa7adf392f77199a6d328d1eabda679b469ebd828e522558
Jul 12 15:23:18.322727 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:18.322638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-dd77f575d-ftchp" event={"ID":"9d6d9ade-f3c1-41a4-a488-993f8d236852","Type":"ContainerStarted","Data":"634e97b1fd8a840bfa7adf392f77199a6d328d1eabda679b469ebd828e522558"}
Jul 12 15:23:19.329123 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:19.329086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-dd77f575d-ftchp" event={"ID":"9d6d9ade-f3c1-41a4-a488-993f8d236852","Type":"ContainerStarted","Data":"7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7"}
Jul 12 15:23:19.346012 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:19.345957    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-dd77f575d-ftchp" podStartSLOduration=2.028048499 podStartE2EDuration="2.345940409s" totalImagesPullingTime="317.89191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:23:19.344550463 +0000 UTC m=+817.712151312" watchObservedRunningTime="2026-07-12 15:23:19.345940409 +0000 UTC m=+817.713541288"
Jul 12 15:23:19.373990 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:19.373956    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2k6r2"]
Jul 12 15:23:19.374179 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:19.374160    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-2k6r2" podUID="58e82fb4-e6b3-4675-804c-9e78b7760bce" containerName="authorino" containerID="cri-o://a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b" gracePeriod=30
Jul 12 15:23:19.632670 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:19.632642    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2k6r2"
Jul 12 15:23:19.740027 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:19.739982    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dvkg\" (UniqueName: \"kubernetes.io/projected/58e82fb4-e6b3-4675-804c-9e78b7760bce-kube-api-access-8dvkg\") pod \"58e82fb4-e6b3-4675-804c-9e78b7760bce\" (UID: \"58e82fb4-e6b3-4675-804c-9e78b7760bce\") "
Jul 12 15:23:19.742186 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:19.742157    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58e82fb4-e6b3-4675-804c-9e78b7760bce-kube-api-access-8dvkg" (OuterVolumeSpecName: "kube-api-access-8dvkg") pod "58e82fb4-e6b3-4675-804c-9e78b7760bce" (UID: "58e82fb4-e6b3-4675-804c-9e78b7760bce"). InnerVolumeSpecName "kube-api-access-8dvkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:23:19.841155 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:19.841120    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dvkg\" (UniqueName: \"kubernetes.io/projected/58e82fb4-e6b3-4675-804c-9e78b7760bce-kube-api-access-8dvkg\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:23:20.335292 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.335261    2578 generic.go:356] "Generic (PLEG): container finished" podID="58e82fb4-e6b3-4675-804c-9e78b7760bce" containerID="a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b" exitCode=0
Jul 12 15:23:20.335768 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.335304    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2k6r2"
Jul 12 15:23:20.335768 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.335343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2k6r2" event={"ID":"58e82fb4-e6b3-4675-804c-9e78b7760bce","Type":"ContainerDied","Data":"a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b"}
Jul 12 15:23:20.335768 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.335380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2k6r2" event={"ID":"58e82fb4-e6b3-4675-804c-9e78b7760bce","Type":"ContainerDied","Data":"a7a8cdb2bc2ef799a438a8dd3b681572e415c1cf223217059be81f083dd329d3"}
Jul 12 15:23:20.335768 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.335397    2578 scope.go:117] "RemoveContainer" containerID="a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b"
Jul 12 15:23:20.343904 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.343885    2578 scope.go:117] "RemoveContainer" containerID="a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b"
Jul 12 15:23:20.344219 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:23:20.344196    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b\": container with ID starting with a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b not found: ID does not exist" containerID="a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b"
Jul 12 15:23:20.344312 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.344226    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b"} err="failed to get container status \"a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b\": rpc error: code = NotFound desc = could not find container \"a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b\": container with ID starting with a783338e3f70fc631c01daf0f8f9d6f27421928b7b29e4045c59c5af592a8b8b not found: ID does not exist"
Jul 12 15:23:20.351511 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.351487    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2k6r2"]
Jul 12 15:23:20.353055 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:20.353034    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2k6r2"]
Jul 12 15:23:22.155732 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:22.155687    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58e82fb4-e6b3-4675-804c-9e78b7760bce" path="/var/lib/kubelet/pods/58e82fb4-e6b3-4675-804c-9e78b7760bce/volumes"
Jul 12 15:23:56.527391 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.527348    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5xhh8"]
Jul 12 15:23:56.527795 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.527725    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58e82fb4-e6b3-4675-804c-9e78b7760bce" containerName="authorino"
Jul 12 15:23:56.527795 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.527737    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="58e82fb4-e6b3-4675-804c-9e78b7760bce" containerName="authorino"
Jul 12 15:23:56.527890 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.527816    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="58e82fb4-e6b3-4675-804c-9e78b7760bce" containerName="authorino"
Jul 12 15:23:56.531845 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.531826    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:56.534355 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.534325    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 12 15:23:56.540552 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.540526    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5xhh8"]
Jul 12 15:23:56.678509 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.678452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62v5l\" (UniqueName: \"kubernetes.io/projected/39b93fb0-0ab9-4374-811e-0fb7228a13fa-kube-api-access-62v5l\") pod \"payload-pre-processing-7964457696-5xhh8\" (UID: \"39b93fb0-0ab9-4374-811e-0fb7228a13fa\") " pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:56.678509 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.678516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39b93fb0-0ab9-4374-811e-0fb7228a13fa-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5xhh8\" (UID: \"39b93fb0-0ab9-4374-811e-0fb7228a13fa\") " pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:56.779504 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.779407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39b93fb0-0ab9-4374-811e-0fb7228a13fa-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5xhh8\" (UID: \"39b93fb0-0ab9-4374-811e-0fb7228a13fa\") " pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:56.779641 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.779518    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62v5l\" (UniqueName: \"kubernetes.io/projected/39b93fb0-0ab9-4374-811e-0fb7228a13fa-kube-api-access-62v5l\") pod \"payload-pre-processing-7964457696-5xhh8\" (UID: \"39b93fb0-0ab9-4374-811e-0fb7228a13fa\") " pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:56.780199 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.780176    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39b93fb0-0ab9-4374-811e-0fb7228a13fa-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5xhh8\" (UID: \"39b93fb0-0ab9-4374-811e-0fb7228a13fa\") " pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:56.787955 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.787924    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62v5l\" (UniqueName: \"kubernetes.io/projected/39b93fb0-0ab9-4374-811e-0fb7228a13fa-kube-api-access-62v5l\") pod \"payload-pre-processing-7964457696-5xhh8\" (UID: \"39b93fb0-0ab9-4374-811e-0fb7228a13fa\") " pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:56.842731 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.842685    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:56.978019 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:56.977991    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5xhh8"]
Jul 12 15:23:56.980771 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:23:56.980736    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b93fb0_0ab9_4374_811e_0fb7228a13fa.slice/crio-29e471ca49649fd3e1a74c7b32faed7e4c291bc58c36d52aaa190b032a2da4bf WatchSource:0}: Error finding container 29e471ca49649fd3e1a74c7b32faed7e4c291bc58c36d52aaa190b032a2da4bf: Status 404 returned error can't find the container with id 29e471ca49649fd3e1a74c7b32faed7e4c291bc58c36d52aaa190b032a2da4bf
Jul 12 15:23:57.515153 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:57.515109    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8" event={"ID":"39b93fb0-0ab9-4374-811e-0fb7228a13fa","Type":"ContainerStarted","Data":"29e471ca49649fd3e1a74c7b32faed7e4c291bc58c36d52aaa190b032a2da4bf"}
Jul 12 15:23:58.521474 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:58.521387    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8" event={"ID":"39b93fb0-0ab9-4374-811e-0fb7228a13fa","Type":"ContainerStarted","Data":"27c6722e4cf8b68210b2a457687a3755c6ae349d30e328ab2446d1bad6360404"}
Jul 12 15:23:58.521876 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:58.521511    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:23:58.535908 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:23:58.535825    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8" podStartSLOduration=1.275199202 podStartE2EDuration="2.535787334s" totalImagesPullingTime="1.260588132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:23:58.535461437 +0000 UTC m=+856.903062311" watchObservedRunningTime="2026-07-12 15:23:58.535787334 +0000 UTC m=+856.903388184"
Jul 12 15:24:09.529145 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:09.529109    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-5xhh8"
Jul 12 15:24:19.125130 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.125087    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"]
Jul 12 15:24:19.128409 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.128391    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.130968 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.130945    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 12 15:24:19.132107 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.132089    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5qs5s\""
Jul 12 15:24:19.132215 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.132125    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 12 15:24:19.132215 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.132164    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 12 15:24:19.137917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.137897    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"]
Jul 12 15:24:19.172577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.172542    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.172577 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.172582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.173026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.172610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9810d92e-8e80-4370-8163-4a85748267cc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.173026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.172707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.173026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.172827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk4mx\" (UniqueName: \"kubernetes.io/projected/9810d92e-8e80-4370-8163-4a85748267cc-kube-api-access-rk4mx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.173026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.172864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.273936 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.273901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9810d92e-8e80-4370-8163-4a85748267cc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.274088 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.273948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.274170 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.274143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk4mx\" (UniqueName: \"kubernetes.io/projected/9810d92e-8e80-4370-8163-4a85748267cc-kube-api-access-rk4mx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.274233 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.274189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.274288 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.274240    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.274288 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.274257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.274393 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.274339    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.274564 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.274538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.274617 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.274596    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.276485 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.276461    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9810d92e-8e80-4370-8163-4a85748267cc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.276592 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.276535    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9810d92e-8e80-4370-8163-4a85748267cc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.281690 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.281669    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk4mx\" (UniqueName: \"kubernetes.io/projected/9810d92e-8e80-4370-8163-4a85748267cc-kube-api-access-rk4mx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7\" (UID: \"9810d92e-8e80-4370-8163-4a85748267cc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.440169 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.440071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:19.586735 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.586686    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"]
Jul 12 15:24:19.590570 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:24:19.590538    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9810d92e_8e80_4370_8163_4a85748267cc.slice/crio-905782cbf1bdce93864dd22e6d1838dcdfe4549e20d6bf95cece759abe8d46f4 WatchSource:0}: Error finding container 905782cbf1bdce93864dd22e6d1838dcdfe4549e20d6bf95cece759abe8d46f4: Status 404 returned error can't find the container with id 905782cbf1bdce93864dd22e6d1838dcdfe4549e20d6bf95cece759abe8d46f4
Jul 12 15:24:19.628685 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:19.628631    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7" event={"ID":"9810d92e-8e80-4370-8163-4a85748267cc","Type":"ContainerStarted","Data":"905782cbf1bdce93864dd22e6d1838dcdfe4549e20d6bf95cece759abe8d46f4"}
Jul 12 15:24:21.642460 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:21.642423    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7" event={"ID":"9810d92e-8e80-4370-8163-4a85748267cc","Type":"ContainerStarted","Data":"b87d8c8a3690fd108ae8389df0cb9ba7e785d047c87451b1c12cf5dc860ae2d1"}
Jul 12 15:24:21.658570 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:21.658517    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7" podStartSLOduration=1.5535450549999998 podStartE2EDuration="2.658498092s" totalImagesPullingTime="1.104953037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:21.658190658 +0000 UTC m=+880.025791506" watchObservedRunningTime="2026-07-12 15:24:21.658498092 +0000 UTC m=+880.026098940"
Jul 12 15:24:26.513555 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.513515    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"]
Jul 12 15:24:26.517717 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.517698    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.520229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.520207    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 12 15:24:26.526622 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.526598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"]
Jul 12 15:24:26.647836 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.647771    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.648014 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.647877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.648014 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.647959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06515ed3-4011-47cb-a5f6-db3799b3c54a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.648089 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.648040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.648089 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.648063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.648165 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.648131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4s4l\" (UniqueName: \"kubernetes.io/projected/06515ed3-4011-47cb-a5f6-db3799b3c54a-kube-api-access-z4s4l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.749433 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.749397    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.749433 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.749437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.749912 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.749470    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4s4l\" (UniqueName: \"kubernetes.io/projected/06515ed3-4011-47cb-a5f6-db3799b3c54a-kube-api-access-z4s4l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.749912 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.749509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.749912 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.749536    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.749912 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.749608    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06515ed3-4011-47cb-a5f6-db3799b3c54a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.750095 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.749999    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.750095 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.750075    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.750209 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.750173    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.752062 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.752038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06515ed3-4011-47cb-a5f6-db3799b3c54a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.752237 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.752219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06515ed3-4011-47cb-a5f6-db3799b3c54a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.758279 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.758249    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4s4l\" (UniqueName: \"kubernetes.io/projected/06515ed3-4011-47cb-a5f6-db3799b3c54a-kube-api-access-z4s4l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kn5\" (UID: \"06515ed3-4011-47cb-a5f6-db3799b3c54a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.829491 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.829402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:26.962902 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:26.962780    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"]
Jul 12 15:24:26.965669 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:24:26.965643    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06515ed3_4011_47cb_a5f6_db3799b3c54a.slice/crio-ca24c0477b2ef012995e90952d9a7ba05edb3d2b029affa7074aae67756ec83e WatchSource:0}: Error finding container ca24c0477b2ef012995e90952d9a7ba05edb3d2b029affa7074aae67756ec83e: Status 404 returned error can't find the container with id ca24c0477b2ef012995e90952d9a7ba05edb3d2b029affa7074aae67756ec83e
Jul 12 15:24:27.675454 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:27.675412    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5" event={"ID":"06515ed3-4011-47cb-a5f6-db3799b3c54a","Type":"ContainerStarted","Data":"53e5a1761e52d98973d299c61f3faaf99aa8919db55930d29ba1121e57c5368c"}
Jul 12 15:24:27.675877 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:27.675479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5" event={"ID":"06515ed3-4011-47cb-a5f6-db3799b3c54a","Type":"ContainerStarted","Data":"ca24c0477b2ef012995e90952d9a7ba05edb3d2b029affa7074aae67756ec83e"}
Jul 12 15:24:27.691897 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:27.691831    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5" podStartSLOduration=1.522873931 podStartE2EDuration="1.69179513s" totalImagesPullingTime="168.921199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:27.690379368 +0000 UTC m=+886.057980215" watchObservedRunningTime="2026-07-12 15:24:27.69179513 +0000 UTC m=+886.059395977"
Jul 12 15:24:29.440855 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:29.440821    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:29.441237 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:29.440908    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:29.453400 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:29.453373    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:29.696492 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:29.696411    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7"
Jul 12 15:24:36.830541 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:36.830501    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:36.830959 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:36.830566    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:36.843025 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:36.842994    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:24:37.733835 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:24:37.733769    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kn5"
Jul 12 15:26:19.997377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:19.997345    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d465c99c9-p4lzv"]
Jul 12 15:26:20.000974 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.000952    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d465c99c9-p4lzv"
Jul 12 15:26:20.010832 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.010788    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d465c99c9-p4lzv"]
Jul 12 15:26:20.089136 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.089099    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whkht\" (UniqueName: \"kubernetes.io/projected/6aed2e46-26b8-46af-b648-6c1151c386fe-kube-api-access-whkht\") pod \"authorino-7d465c99c9-p4lzv\" (UID: \"6aed2e46-26b8-46af-b648-6c1151c386fe\") " pod="kuadrant-system/authorino-7d465c99c9-p4lzv"
Jul 12 15:26:20.089351 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.089180    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6aed2e46-26b8-46af-b648-6c1151c386fe-tls-cert\") pod \"authorino-7d465c99c9-p4lzv\" (UID: \"6aed2e46-26b8-46af-b648-6c1151c386fe\") " pod="kuadrant-system/authorino-7d465c99c9-p4lzv"
Jul 12 15:26:20.189852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.189782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whkht\" (UniqueName: \"kubernetes.io/projected/6aed2e46-26b8-46af-b648-6c1151c386fe-kube-api-access-whkht\") pod \"authorino-7d465c99c9-p4lzv\" (UID: \"6aed2e46-26b8-46af-b648-6c1151c386fe\") " pod="kuadrant-system/authorino-7d465c99c9-p4lzv"
Jul 12 15:26:20.190052 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.189890    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6aed2e46-26b8-46af-b648-6c1151c386fe-tls-cert\") pod \"authorino-7d465c99c9-p4lzv\" (UID: \"6aed2e46-26b8-46af-b648-6c1151c386fe\") " pod="kuadrant-system/authorino-7d465c99c9-p4lzv"
Jul 12 15:26:20.192619 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.192589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6aed2e46-26b8-46af-b648-6c1151c386fe-tls-cert\") pod \"authorino-7d465c99c9-p4lzv\" (UID: \"6aed2e46-26b8-46af-b648-6c1151c386fe\") " pod="kuadrant-system/authorino-7d465c99c9-p4lzv"
Jul 12 15:26:20.197475 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.197448    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whkht\" (UniqueName: \"kubernetes.io/projected/6aed2e46-26b8-46af-b648-6c1151c386fe-kube-api-access-whkht\") pod \"authorino-7d465c99c9-p4lzv\" (UID: \"6aed2e46-26b8-46af-b648-6c1151c386fe\") " pod="kuadrant-system/authorino-7d465c99c9-p4lzv"
Jul 12 15:26:20.310886 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.310759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d465c99c9-p4lzv"
Jul 12 15:26:20.441597 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.441575    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d465c99c9-p4lzv"]
Jul 12 15:26:20.443555 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:26:20.443528    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6aed2e46_26b8_46af_b648_6c1151c386fe.slice/crio-592d762b2c817646cd2ee730bfbdbbbd234a72235590eb2eadaad51a4f64fd9d WatchSource:0}: Error finding container 592d762b2c817646cd2ee730bfbdbbbd234a72235590eb2eadaad51a4f64fd9d: Status 404 returned error can't find the container with id 592d762b2c817646cd2ee730bfbdbbbd234a72235590eb2eadaad51a4f64fd9d
Jul 12 15:26:20.444993 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:20.444972    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 15:26:21.230300 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.230165    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d465c99c9-p4lzv" event={"ID":"6aed2e46-26b8-46af-b648-6c1151c386fe","Type":"ContainerStarted","Data":"817ac43fe9bc7691992ec43bf029deebba6a178f6ca66bab67db2b387afd4c5e"}
Jul 12 15:26:21.230300 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.230252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d465c99c9-p4lzv" event={"ID":"6aed2e46-26b8-46af-b648-6c1151c386fe","Type":"ContainerStarted","Data":"592d762b2c817646cd2ee730bfbdbbbd234a72235590eb2eadaad51a4f64fd9d"}
Jul 12 15:26:21.247609 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.247548    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d465c99c9-p4lzv" podStartSLOduration=1.7873812089999999 podStartE2EDuration="2.247521128s" totalImagesPullingTime="460.139919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:26:21.246054811 +0000 UTC m=+999.613655659" watchObservedRunningTime="2026-07-12 15:26:21.247521128 +0000 UTC m=+999.615121976"
Jul 12 15:26:21.274248 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.274194    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-dd77f575d-ftchp"]
Jul 12 15:26:21.274478 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.274434    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-dd77f575d-ftchp" podUID="9d6d9ade-f3c1-41a4-a488-993f8d236852" containerName="authorino" containerID="cri-o://7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7" gracePeriod=30
Jul 12 15:26:21.549710 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.549680    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:26:21.603649 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.603609    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psgrg\" (UniqueName: \"kubernetes.io/projected/9d6d9ade-f3c1-41a4-a488-993f8d236852-kube-api-access-psgrg\") pod \"9d6d9ade-f3c1-41a4-a488-993f8d236852\" (UID: \"9d6d9ade-f3c1-41a4-a488-993f8d236852\") "
Jul 12 15:26:21.603844 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.603730    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d6d9ade-f3c1-41a4-a488-993f8d236852-tls-cert\") pod \"9d6d9ade-f3c1-41a4-a488-993f8d236852\" (UID: \"9d6d9ade-f3c1-41a4-a488-993f8d236852\") "
Jul 12 15:26:21.605866 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.605795    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6d9ade-f3c1-41a4-a488-993f8d236852-kube-api-access-psgrg" (OuterVolumeSpecName: "kube-api-access-psgrg") pod "9d6d9ade-f3c1-41a4-a488-993f8d236852" (UID: "9d6d9ade-f3c1-41a4-a488-993f8d236852"). InnerVolumeSpecName "kube-api-access-psgrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:26:21.620601 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.619841    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6d9ade-f3c1-41a4-a488-993f8d236852-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9d6d9ade-f3c1-41a4-a488-993f8d236852" (UID: "9d6d9ade-f3c1-41a4-a488-993f8d236852"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:26:21.705251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.705214    2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d6d9ade-f3c1-41a4-a488-993f8d236852-tls-cert\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:26:21.705251 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:21.705244    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psgrg\" (UniqueName: \"kubernetes.io/projected/9d6d9ade-f3c1-41a4-a488-993f8d236852-kube-api-access-psgrg\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:26:22.237944 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.237900    2578 generic.go:356] "Generic (PLEG): container finished" podID="9d6d9ade-f3c1-41a4-a488-993f8d236852" containerID="7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7" exitCode=0
Jul 12 15:26:22.238411 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.237973    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dd77f575d-ftchp"
Jul 12 15:26:22.238411 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.237988    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-dd77f575d-ftchp" event={"ID":"9d6d9ade-f3c1-41a4-a488-993f8d236852","Type":"ContainerDied","Data":"7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7"}
Jul 12 15:26:22.238411 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.238046    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-dd77f575d-ftchp" event={"ID":"9d6d9ade-f3c1-41a4-a488-993f8d236852","Type":"ContainerDied","Data":"634e97b1fd8a840bfa7adf392f77199a6d328d1eabda679b469ebd828e522558"}
Jul 12 15:26:22.238411 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.238071    2578 scope.go:117] "RemoveContainer" containerID="7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7"
Jul 12 15:26:22.249958 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.249929    2578 scope.go:117] "RemoveContainer" containerID="7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7"
Jul 12 15:26:22.250308 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:26:22.250279    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7\": container with ID starting with 7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7 not found: ID does not exist" containerID="7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7"
Jul 12 15:26:22.250394 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.250322    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7"} err="failed to get container status \"7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7\": rpc error: code = NotFound desc = could not find container \"7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7\": container with ID starting with 7638771b5975f735bcf521b665954dee6447b50449424b9d13cec4e62ac943d7 not found: ID does not exist"
Jul 12 15:26:22.257017 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.256985    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-dd77f575d-ftchp"]
Jul 12 15:26:22.258707 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:22.258684    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-dd77f575d-ftchp"]
Jul 12 15:26:24.154875 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:26:24.154842    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d6d9ade-f3c1-41a4-a488-993f8d236852" path="/var/lib/kubelet/pods/9d6d9ade-f3c1-41a4-a488-993f8d236852/volumes"
Jul 12 15:42:50.320603 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.320569    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"]
Jul 12 15:42:50.321124 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.320963    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6d9ade-f3c1-41a4-a488-993f8d236852" containerName="authorino"
Jul 12 15:42:50.321124 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.320974    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6d9ade-f3c1-41a4-a488-993f8d236852" containerName="authorino"
Jul 12 15:42:50.321124 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.321058    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6d9ade-f3c1-41a4-a488-993f8d236852" containerName="authorino"
Jul 12 15:42:50.324320 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.324302    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.327046 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.327015    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ffed116f\"/\"e2e-lifecycle-model-ffed116f-kserve-self-signed-certs\""
Jul 12 15:42:50.327046 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.327041    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ffed116f\"/\"openshift-service-ca.crt\""
Jul 12 15:42:50.327223 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.327020    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ffed116f\"/\"default-dockercfg-przlz\""
Jul 12 15:42:50.328122 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.328093    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ffed116f\"/\"kube-root-ca.crt\""
Jul 12 15:42:50.332478 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.332457    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"]
Jul 12 15:42:50.455424 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.455377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwdj6\" (UniqueName: \"kubernetes.io/projected/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-kube-api-access-pwdj6\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.455605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.455439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-model-cache\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.455605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.455488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tls-certs\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.455605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.455513    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tmp-dir\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.455605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.455567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-home\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.455745 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.455614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-dshm\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.556638 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.556592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tls-certs\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.556890 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.556648    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tmp-dir\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.556890 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.556683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-home\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.556890 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.556744    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-dshm\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.556890 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.556799    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwdj6\" (UniqueName: \"kubernetes.io/projected/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-kube-api-access-pwdj6\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.556890 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.556884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-model-cache\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.557186 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.557161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tmp-dir\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.557373 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.557206    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-home\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.557373 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.557273    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-model-cache\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.559092 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.559064    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-dshm\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.559317 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.559300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tls-certs\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.564578 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.564555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwdj6\" (UniqueName: \"kubernetes.io/projected/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-kube-api-access-pwdj6\") pod \"e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") " pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.634696 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.634651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:42:50.774799 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.774765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"]
Jul 12 15:42:50.776990 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:42:50.776961    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda55b3e1a_2255_423b_bc1b_acb1bd91ba7b.slice/crio-c84b6f0650d9e45cec811455c3cf1fa7585c552caae7de2a3d715045135fcf01 WatchSource:0}: Error finding container c84b6f0650d9e45cec811455c3cf1fa7585c552caae7de2a3d715045135fcf01: Status 404 returned error can't find the container with id c84b6f0650d9e45cec811455c3cf1fa7585c552caae7de2a3d715045135fcf01
Jul 12 15:42:50.982654 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.982553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw" event={"ID":"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b","Type":"ContainerStarted","Data":"e22f0b37926e4b4d46c0f5e7fe5dad492c1078b2718ca40b603c4c7a05b339de"}
Jul 12 15:42:50.982654 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.982603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw" event={"ID":"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b","Type":"ContainerStarted","Data":"c84b6f0650d9e45cec811455c3cf1fa7585c552caae7de2a3d715045135fcf01"}
Jul 12 15:42:50.998641 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:42:50.998589    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw" podStartSLOduration=0.998569773 podStartE2EDuration="998.569773ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:42:50.997327098 +0000 UTC m=+1989.364927963" watchObservedRunningTime="2026-07-12 15:42:50.998569773 +0000 UTC m=+1989.366170619"
Jul 12 15:43:00.635765 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:00.635706    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:43:00.635765 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:00.635786    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:43:00.648514 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:00.648482    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:43:01.041898 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:01.041792    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:43:25.039378 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:25.039343    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"]
Jul 12 15:43:40.060035 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.059972    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw" podUID="a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" containerName="main" containerID="cri-o://e22f0b37926e4b4d46c0f5e7fe5dad492c1078b2718ca40b603c4c7a05b339de" gracePeriod=45
Jul 12 15:43:40.219783 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.219752    2578 generic.go:356] "Generic (PLEG): container finished" podID="a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" containerID="e22f0b37926e4b4d46c0f5e7fe5dad492c1078b2718ca40b603c4c7a05b339de" exitCode=0
Jul 12 15:43:40.219987 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.219843    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw" event={"ID":"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b","Type":"ContainerDied","Data":"e22f0b37926e4b4d46c0f5e7fe5dad492c1078b2718ca40b603c4c7a05b339de"}
Jul 12 15:43:40.306053 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.306023    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:43:40.339798 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.339774    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-dshm\") pod \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") "
Jul 12 15:43:40.339989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.339832    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tmp-dir\") pod \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") "
Jul 12 15:43:40.339989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.339908    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-home\") pod \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") "
Jul 12 15:43:40.339989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.339927    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tls-certs\") pod \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") "
Jul 12 15:43:40.339989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.339955    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwdj6\" (UniqueName: \"kubernetes.io/projected/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-kube-api-access-pwdj6\") pod \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") "
Jul 12 15:43:40.339989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.339973    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-model-cache\") pod \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\" (UID: \"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b\") "
Jul 12 15:43:40.340269 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.340153    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-home" (OuterVolumeSpecName: "home") pod "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" (UID: "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:43:40.340269 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.340165    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" (UID: "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:43:40.340269 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.340253    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:43:40.340496 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.340282    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:43:40.340496 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.340344    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" (UID: "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:43:40.342220 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.342195    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-dshm" (OuterVolumeSpecName: "dshm") pod "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" (UID: "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:43:40.342515 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.342485    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" (UID: "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:43:40.342515 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.342495    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-kube-api-access-pwdj6" (OuterVolumeSpecName: "kube-api-access-pwdj6") pod "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" (UID: "a55b3e1a-2255-423b-bc1b-acb1bd91ba7b"). InnerVolumeSpecName "kube-api-access-pwdj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:43:40.441236 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.441190    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:43:40.441236 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.441226    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwdj6\" (UniqueName: \"kubernetes.io/projected/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-kube-api-access-pwdj6\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:43:40.441236 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.441239    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:43:40.441236 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:40.441248    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:43:41.226786 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:41.226741    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"
Jul 12 15:43:41.226786 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:41.226757    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw" event={"ID":"a55b3e1a-2255-423b-bc1b-acb1bd91ba7b","Type":"ContainerDied","Data":"c84b6f0650d9e45cec811455c3cf1fa7585c552caae7de2a3d715045135fcf01"}
Jul 12 15:43:41.227255 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:41.226830    2578 scope.go:117] "RemoveContainer" containerID="e22f0b37926e4b4d46c0f5e7fe5dad492c1078b2718ca40b603c4c7a05b339de"
Jul 12 15:43:41.247898 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:41.247874    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"]
Jul 12 15:43:41.249534 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:41.249514    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ffed116f/e2e-lifecycle-model-ffed116f-kserve-b94847855-6bmpw"]
Jul 12 15:43:42.156362 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:43:42.156333    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" path="/var/lib/kubelet/pods/a55b3e1a-2255-423b-bc1b-acb1bd91ba7b/volumes"
Jul 12 15:45:25.716273 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.716240    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"]
Jul 12 15:45:25.716857 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.716837    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" containerName="main"
Jul 12 15:45:25.716912 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.716860    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" containerName="main"
Jul 12 15:45:25.716985 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.716974    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a55b3e1a-2255-423b-bc1b-acb1bd91ba7b" containerName="main"
Jul 12 15:45:25.720012 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.719998    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.722901 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.722883    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-eba8e3\"/\"openshift-service-ca.crt\""
Jul 12 15:45:25.724553 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.724248    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-eba8e3\"/\"kube-root-ca.crt\""
Jul 12 15:45:25.724553 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.724283    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-eba8e3\"/\"default-dockercfg-7dnwn\""
Jul 12 15:45:25.724553 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.724249    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-eba8e3\"/\"test-model-eba8e3-kserve-self-signed-certs\""
Jul 12 15:45:25.726586 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.726566    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"]
Jul 12 15:45:25.846735 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.846709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llc77\" (UniqueName: \"kubernetes.io/projected/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-kube-api-access-llc77\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.846875 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.846752    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-dshm\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.846875 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.846833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tls-certs\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.847001 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.846908    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-model-cache\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.847001 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.846960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-home\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.847094 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.847010    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tmp-dir\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.947470 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.947426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llc77\" (UniqueName: \"kubernetes.io/projected/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-kube-api-access-llc77\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.947611 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.947485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-dshm\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.947611 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.947519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tls-certs\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.947611 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.947540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-model-cache\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.947611 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.947561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-home\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.947611 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.947598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tmp-dir\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.947983 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.947962    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-model-cache\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.948057 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.948030    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tmp-dir\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.948139 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.948061    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-home\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.949943 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.949922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-dshm\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.950075 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.950060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tls-certs\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:25.954428 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:25.954405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llc77\" (UniqueName: \"kubernetes.io/projected/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-kube-api-access-llc77\") pod \"test-model-eba8e3-kserve-7d7c8b6d77-m7vgb\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") " pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:26.029963 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:26.029891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:26.155859 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:26.155835    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"]
Jul 12 15:45:26.157108 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:45:26.157085    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e356d0e_bbca_4b63_a3a0_853e1b9f4319.slice/crio-ce54e60551ffe66cfaa3fa8ba872879a998203ede5da1d048ab3d680aa6bc4d4 WatchSource:0}: Error finding container ce54e60551ffe66cfaa3fa8ba872879a998203ede5da1d048ab3d680aa6bc4d4: Status 404 returned error can't find the container with id ce54e60551ffe66cfaa3fa8ba872879a998203ede5da1d048ab3d680aa6bc4d4
Jul 12 15:45:26.715417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:26.715375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb" event={"ID":"7e356d0e-bbca-4b63-a3a0-853e1b9f4319","Type":"ContainerStarted","Data":"59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21"}
Jul 12 15:45:26.715417 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:26.715421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb" event={"ID":"7e356d0e-bbca-4b63-a3a0-853e1b9f4319","Type":"ContainerStarted","Data":"ce54e60551ffe66cfaa3fa8ba872879a998203ede5da1d048ab3d680aa6bc4d4"}
Jul 12 15:45:26.732920 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:26.732873    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb" podStartSLOduration=1.732857358 podStartE2EDuration="1.732857358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:45:26.732380869 +0000 UTC m=+2145.099981721" watchObservedRunningTime="2026-07-12 15:45:26.732857358 +0000 UTC m=+2145.100458204"
Jul 12 15:45:36.030157 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:36.030123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:36.030734 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:36.030172    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:36.042683 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:36.042658    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:36.780511 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:36.780483    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:45:55.000524 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.000489    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"]
Jul 12 15:45:55.004046 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.004025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.007619 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.007593    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6cab7d\"/\"kube-root-ca.crt\""
Jul 12 15:45:55.007758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.007593    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6cab7d\"/\"default-dockercfg-phsmp\""
Jul 12 15:45:55.007758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.007660    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6cab7d\"/\"test-model-6cab7d-kserve-self-signed-certs\""
Jul 12 15:45:55.007758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.007703    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6cab7d\"/\"openshift-service-ca.crt\""
Jul 12 15:45:55.011474 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.011451    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"]
Jul 12 15:45:55.113494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.113430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgtl4\" (UniqueName: \"kubernetes.io/projected/efaff7e4-31c6-4881-9405-2d290fa75bde-kube-api-access-xgtl4\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.113660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.113545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efaff7e4-31c6-4881-9405-2d290fa75bde-tls-certs\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.113660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.113599    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-dshm\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.113660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.113619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-model-cache\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.113825 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.113682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-home\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.113825 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.113788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-tmp-dir\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.214526 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.214489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-tmp-dir\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.214729 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.214544    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgtl4\" (UniqueName: \"kubernetes.io/projected/efaff7e4-31c6-4881-9405-2d290fa75bde-kube-api-access-xgtl4\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.214729 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.214586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efaff7e4-31c6-4881-9405-2d290fa75bde-tls-certs\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.214729 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.214618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-dshm\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.214729 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.214636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-model-cache\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.214729 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.214690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-home\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.215038 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.214982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-tmp-dir\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.215095 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.215074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-model-cache\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.215208 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.215183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-home\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.216915 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.216891    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-dshm\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.217954 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.217932    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efaff7e4-31c6-4881-9405-2d290fa75bde-tls-certs\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.222661 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.222638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgtl4\" (UniqueName: \"kubernetes.io/projected/efaff7e4-31c6-4881-9405-2d290fa75bde-kube-api-access-xgtl4\") pod \"test-model-6cab7d-kserve-75bf55bdff-dphd6\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") " pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.314913 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.314788    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:45:55.445457 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.445433    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"]
Jul 12 15:45:55.447717 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:45:55.447680    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefaff7e4_31c6_4881_9405_2d290fa75bde.slice/crio-062eb91a4247bcca57890b74e7208d382b40da761acfc619fc198e9c0d1a0554 WatchSource:0}: Error finding container 062eb91a4247bcca57890b74e7208d382b40da761acfc619fc198e9c0d1a0554: Status 404 returned error can't find the container with id 062eb91a4247bcca57890b74e7208d382b40da761acfc619fc198e9c0d1a0554
Jul 12 15:45:55.859119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.859084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6" event={"ID":"efaff7e4-31c6-4881-9405-2d290fa75bde","Type":"ContainerStarted","Data":"35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40"}
Jul 12 15:45:55.859119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.859126    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6" event={"ID":"efaff7e4-31c6-4881-9405-2d290fa75bde","Type":"ContainerStarted","Data":"062eb91a4247bcca57890b74e7208d382b40da761acfc619fc198e9c0d1a0554"}
Jul 12 15:45:55.876340 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:45:55.876293    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6" podStartSLOduration=1.8762766690000001 podStartE2EDuration="1.876276669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:45:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:45:55.875231369 +0000 UTC m=+2174.242832218" watchObservedRunningTime="2026-07-12 15:45:55.876276669 +0000 UTC m=+2174.243877516"
Jul 12 15:46:05.315081 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:05.315039    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:46:05.315557 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:05.315093    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:46:05.333981 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:05.333949    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:46:05.922890 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:05.922844    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:46:31.495584 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:31.495551    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"]
Jul 12 15:46:32.344732 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:32.344700    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"]
Jul 12 15:46:46.518113 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.518050    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb" podUID="7e356d0e-bbca-4b63-a3a0-853e1b9f4319" containerName="main" containerID="cri-o://59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21" gracePeriod=45
Jul 12 15:46:46.758452 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.758430    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:46:46.896447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896420    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tls-certs\") pod \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") "
Jul 12 15:46:46.896604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896453    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-home\") pod \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") "
Jul 12 15:46:46.896604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896544    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-model-cache\") pod \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") "
Jul 12 15:46:46.896604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896575    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-dshm\") pod \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") "
Jul 12 15:46:46.896604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896596    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tmp-dir\") pod \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") "
Jul 12 15:46:46.896836 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896615    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llc77\" (UniqueName: \"kubernetes.io/projected/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-kube-api-access-llc77\") pod \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\" (UID: \"7e356d0e-bbca-4b63-a3a0-853e1b9f4319\") "
Jul 12 15:46:46.896836 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896762    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-home" (OuterVolumeSpecName: "home") pod "7e356d0e-bbca-4b63-a3a0-853e1b9f4319" (UID: "7e356d0e-bbca-4b63-a3a0-853e1b9f4319"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:46:46.896957 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896895    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e356d0e-bbca-4b63-a3a0-853e1b9f4319" (UID: "7e356d0e-bbca-4b63-a3a0-853e1b9f4319"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:46:46.896957 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896915    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e356d0e-bbca-4b63-a3a0-853e1b9f4319" (UID: "7e356d0e-bbca-4b63-a3a0-853e1b9f4319"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:46:46.896957 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.896943    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:46.898800 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.898775    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e356d0e-bbca-4b63-a3a0-853e1b9f4319" (UID: "7e356d0e-bbca-4b63-a3a0-853e1b9f4319"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:46:46.898800 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.898773    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-kube-api-access-llc77" (OuterVolumeSpecName: "kube-api-access-llc77") pod "7e356d0e-bbca-4b63-a3a0-853e1b9f4319" (UID: "7e356d0e-bbca-4b63-a3a0-853e1b9f4319"). InnerVolumeSpecName "kube-api-access-llc77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:46:46.898975 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.898834    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-dshm" (OuterVolumeSpecName: "dshm") pod "7e356d0e-bbca-4b63-a3a0-853e1b9f4319" (UID: "7e356d0e-bbca-4b63-a3a0-853e1b9f4319"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:46:46.998191 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.998164    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:46.998191 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.998187    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:46.998191 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.998196    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:46.998377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.998205    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llc77\" (UniqueName: \"kubernetes.io/projected/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-kube-api-access-llc77\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:46.998377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:46.998214    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e356d0e-bbca-4b63-a3a0-853e1b9f4319-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:47.107242 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.107210    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e356d0e-bbca-4b63-a3a0-853e1b9f4319" containerID="59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21" exitCode=0
Jul 12 15:46:47.107443 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.107272    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"
Jul 12 15:46:47.107443 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.107278    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb" event={"ID":"7e356d0e-bbca-4b63-a3a0-853e1b9f4319","Type":"ContainerDied","Data":"59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21"}
Jul 12 15:46:47.107443 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.107317    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb" event={"ID":"7e356d0e-bbca-4b63-a3a0-853e1b9f4319","Type":"ContainerDied","Data":"ce54e60551ffe66cfaa3fa8ba872879a998203ede5da1d048ab3d680aa6bc4d4"}
Jul 12 15:46:47.107443 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.107340    2578 scope.go:117] "RemoveContainer" containerID="59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21"
Jul 12 15:46:47.116636 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.116622    2578 scope.go:117] "RemoveContainer" containerID="59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21"
Jul 12 15:46:47.116930 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:46:47.116910    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21\": container with ID starting with 59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21 not found: ID does not exist" containerID="59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21"
Jul 12 15:46:47.117024 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.116937    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21"} err="failed to get container status \"59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21\": rpc error: code = NotFound desc = could not find container \"59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21\": container with ID starting with 59fa1bee7122fe7cc80781dcab564beb39883b2b6b91537c60b4bdc43af23f21 not found: ID does not exist"
Jul 12 15:46:47.128663 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.128635    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"]
Jul 12 15:46:47.130375 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.130357    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-eba8e3/test-model-eba8e3-kserve-7d7c8b6d77-m7vgb"]
Jul 12 15:46:47.365245 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.365208    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6" podUID="efaff7e4-31c6-4881-9405-2d290fa75bde" containerName="main" containerID="cri-o://35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40" gracePeriod=45
Jul 12 15:46:47.629250 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.629190    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:46:47.805176 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805151    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-model-cache\") pod \"efaff7e4-31c6-4881-9405-2d290fa75bde\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") "
Jul 12 15:46:47.805354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805205    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-home\") pod \"efaff7e4-31c6-4881-9405-2d290fa75bde\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") "
Jul 12 15:46:47.805354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805222    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-tmp-dir\") pod \"efaff7e4-31c6-4881-9405-2d290fa75bde\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") "
Jul 12 15:46:47.805354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805299    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgtl4\" (UniqueName: \"kubernetes.io/projected/efaff7e4-31c6-4881-9405-2d290fa75bde-kube-api-access-xgtl4\") pod \"efaff7e4-31c6-4881-9405-2d290fa75bde\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") "
Jul 12 15:46:47.805354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805348    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efaff7e4-31c6-4881-9405-2d290fa75bde-tls-certs\") pod \"efaff7e4-31c6-4881-9405-2d290fa75bde\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") "
Jul 12 15:46:47.805566 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805373    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-dshm\") pod \"efaff7e4-31c6-4881-9405-2d290fa75bde\" (UID: \"efaff7e4-31c6-4881-9405-2d290fa75bde\") "
Jul 12 15:46:47.805566 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805423    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-model-cache" (OuterVolumeSpecName: "model-cache") pod "efaff7e4-31c6-4881-9405-2d290fa75bde" (UID: "efaff7e4-31c6-4881-9405-2d290fa75bde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:46:47.805566 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805447    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-home" (OuterVolumeSpecName: "home") pod "efaff7e4-31c6-4881-9405-2d290fa75bde" (UID: "efaff7e4-31c6-4881-9405-2d290fa75bde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:46:47.805566 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805512    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "efaff7e4-31c6-4881-9405-2d290fa75bde" (UID: "efaff7e4-31c6-4881-9405-2d290fa75bde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:46:47.805712 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805638    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:47.805712 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805648    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:47.805712 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.805658    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:47.807557 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.807534    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-dshm" (OuterVolumeSpecName: "dshm") pod "efaff7e4-31c6-4881-9405-2d290fa75bde" (UID: "efaff7e4-31c6-4881-9405-2d290fa75bde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:46:47.807557 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.807546    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efaff7e4-31c6-4881-9405-2d290fa75bde-kube-api-access-xgtl4" (OuterVolumeSpecName: "kube-api-access-xgtl4") pod "efaff7e4-31c6-4881-9405-2d290fa75bde" (UID: "efaff7e4-31c6-4881-9405-2d290fa75bde"). InnerVolumeSpecName "kube-api-access-xgtl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:46:47.807722 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.807575    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efaff7e4-31c6-4881-9405-2d290fa75bde-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "efaff7e4-31c6-4881-9405-2d290fa75bde" (UID: "efaff7e4-31c6-4881-9405-2d290fa75bde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:46:47.906502 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.906443    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgtl4\" (UniqueName: \"kubernetes.io/projected/efaff7e4-31c6-4881-9405-2d290fa75bde-kube-api-access-xgtl4\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:47.906502 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.906465    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efaff7e4-31c6-4881-9405-2d290fa75bde-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:47.906502 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:47.906473    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efaff7e4-31c6-4881-9405-2d290fa75bde-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:46:48.114099 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.114071    2578 generic.go:356] "Generic (PLEG): container finished" podID="efaff7e4-31c6-4881-9405-2d290fa75bde" containerID="35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40" exitCode=0
Jul 12 15:46:48.114279 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.114114    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"
Jul 12 15:46:48.114279 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.114166    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6" event={"ID":"efaff7e4-31c6-4881-9405-2d290fa75bde","Type":"ContainerDied","Data":"35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40"}
Jul 12 15:46:48.114279 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.114215    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6" event={"ID":"efaff7e4-31c6-4881-9405-2d290fa75bde","Type":"ContainerDied","Data":"062eb91a4247bcca57890b74e7208d382b40da761acfc619fc198e9c0d1a0554"}
Jul 12 15:46:48.114279 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.114238    2578 scope.go:117] "RemoveContainer" containerID="35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40"
Jul 12 15:46:48.123763 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.123747    2578 scope.go:117] "RemoveContainer" containerID="35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40"
Jul 12 15:46:48.124089 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:46:48.124066    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40\": container with ID starting with 35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40 not found: ID does not exist" containerID="35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40"
Jul 12 15:46:48.124169 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.124096    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40"} err="failed to get container status \"35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40\": rpc error: code = NotFound desc = could not find container \"35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40\": container with ID starting with 35b4fb0e6a9d4aa2a7952a0900f5e73713a14ec11eb6f063288342cb56faef40 not found: ID does not exist"
Jul 12 15:46:48.134221 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.134195    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"]
Jul 12 15:46:48.137749 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.137730    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-6cab7d/test-model-6cab7d-kserve-75bf55bdff-dphd6"]
Jul 12 15:46:48.155346 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.155323    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e356d0e-bbca-4b63-a3a0-853e1b9f4319" path="/var/lib/kubelet/pods/7e356d0e-bbca-4b63-a3a0-853e1b9f4319/volumes"
Jul 12 15:46:48.155699 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:46:48.155686    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efaff7e4-31c6-4881-9405-2d290fa75bde" path="/var/lib/kubelet/pods/efaff7e4-31c6-4881-9405-2d290fa75bde/volumes"
Jul 12 15:47:34.198781 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.198700    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"]
Jul 12 15:47:34.199354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.199302    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efaff7e4-31c6-4881-9405-2d290fa75bde" containerName="main"
Jul 12 15:47:34.199354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.199324    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="efaff7e4-31c6-4881-9405-2d290fa75bde" containerName="main"
Jul 12 15:47:34.199354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.199348    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e356d0e-bbca-4b63-a3a0-853e1b9f4319" containerName="main"
Jul 12 15:47:34.199522 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.199357    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e356d0e-bbca-4b63-a3a0-853e1b9f4319" containerName="main"
Jul 12 15:47:34.199522 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.199457    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e356d0e-bbca-4b63-a3a0-853e1b9f4319" containerName="main"
Jul 12 15:47:34.199522 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.199472    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="efaff7e4-31c6-4881-9405-2d290fa75bde" containerName="main"
Jul 12 15:47:34.202621 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.202598    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.205218 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.205194    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-c20960-serving-cert\""
Jul 12 15:47:34.205353 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.205304    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-lcn4n\""
Jul 12 15:47:34.212360 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.212339    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"]
Jul 12 15:47:34.308829 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.308773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0375c51e-61a9-420d-9bdf-73e628ba550c-openshift-service-ca\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.309007 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.308842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.309007 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.308956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw9nt\" (UniqueName: \"kubernetes.io/projected/0375c51e-61a9-420d-9bdf-73e628ba550c-kube-api-access-qw9nt\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.410359 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.410317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0375c51e-61a9-420d-9bdf-73e628ba550c-openshift-service-ca\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.410564 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.410372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.410564 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.410408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw9nt\" (UniqueName: \"kubernetes.io/projected/0375c51e-61a9-420d-9bdf-73e628ba550c-kube-api-access-qw9nt\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.410564 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:47:34.410522    2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-c20960-serving-cert: secret "maas-api-e2e-api-b-c20960-serving-cert" not found
Jul 12 15:47:34.410752 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:47:34.410598    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls podName:0375c51e-61a9-420d-9bdf-73e628ba550c nodeName:}" failed. No retries permitted until 2026-07-12 15:47:34.91058187 +0000 UTC m=+2273.278182696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls") pod "maas-api-e2e-api-b-c20960-5d9459649-6vl47" (UID: "0375c51e-61a9-420d-9bdf-73e628ba550c") : secret "maas-api-e2e-api-b-c20960-serving-cert" not found
Jul 12 15:47:34.411122 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.411100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0375c51e-61a9-420d-9bdf-73e628ba550c-openshift-service-ca\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.424295 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.424268    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw9nt\" (UniqueName: \"kubernetes.io/projected/0375c51e-61a9-420d-9bdf-73e628ba550c-kube-api-access-qw9nt\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.916459 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:34.916417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:34.916647 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:47:34.916579    2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-c20960-serving-cert: secret "maas-api-e2e-api-b-c20960-serving-cert" not found
Jul 12 15:47:34.916696 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:47:34.916650    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls podName:0375c51e-61a9-420d-9bdf-73e628ba550c nodeName:}" failed. No retries permitted until 2026-07-12 15:47:35.916633486 +0000 UTC m=+2274.284234312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls") pod "maas-api-e2e-api-b-c20960-5d9459649-6vl47" (UID: "0375c51e-61a9-420d-9bdf-73e628ba550c") : secret "maas-api-e2e-api-b-c20960-serving-cert" not found
Jul 12 15:47:35.928030 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:35.927982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:35.930505 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:35.930482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls\") pod \"maas-api-e2e-api-b-c20960-5d9459649-6vl47\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") " pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:36.014370 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:36.014333    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:36.163631 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:36.163609    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"]
Jul 12 15:47:36.166451 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:47:36.166418    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0375c51e_61a9_420d_9bdf_73e628ba550c.slice/crio-41beeb600087fd9184a3645a34f27397eb566dabeb834120f55ae2fefbd523a5 WatchSource:0}: Error finding container 41beeb600087fd9184a3645a34f27397eb566dabeb834120f55ae2fefbd523a5: Status 404 returned error can't find the container with id 41beeb600087fd9184a3645a34f27397eb566dabeb834120f55ae2fefbd523a5
Jul 12 15:47:36.167750 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:36.167729    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 15:47:36.349126 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:36.349085    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47" event={"ID":"0375c51e-61a9-420d-9bdf-73e628ba550c","Type":"ContainerStarted","Data":"41beeb600087fd9184a3645a34f27397eb566dabeb834120f55ae2fefbd523a5"}
Jul 12 15:47:39.365767 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:39.365726    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47" event={"ID":"0375c51e-61a9-420d-9bdf-73e628ba550c","Type":"ContainerStarted","Data":"53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af"}
Jul 12 15:47:39.366191 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:39.365842    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:39.383409 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:39.383359    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47" podStartSLOduration=3.227991008 podStartE2EDuration="5.383343048s" totalImagesPullingTime="2.15535204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:47:39.380935661 +0000 UTC m=+2277.748536509" watchObservedRunningTime="2026-07-12 15:47:39.383343048 +0000 UTC m=+2277.750943894"
Jul 12 15:47:45.376081 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:45.376043    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:59.094323 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.094290    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"]
Jul 12 15:47:59.094796 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.094536    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47" podUID="0375c51e-61a9-420d-9bdf-73e628ba550c" containerName="maas-api" containerID="cri-o://53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af" gracePeriod=30
Jul 12 15:47:59.345554 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.345491    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:59.443279 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.443235    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw9nt\" (UniqueName: \"kubernetes.io/projected/0375c51e-61a9-420d-9bdf-73e628ba550c-kube-api-access-qw9nt\") pod \"0375c51e-61a9-420d-9bdf-73e628ba550c\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") "
Jul 12 15:47:59.443459 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.443334    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls\") pod \"0375c51e-61a9-420d-9bdf-73e628ba550c\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") "
Jul 12 15:47:59.443459 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.443389    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0375c51e-61a9-420d-9bdf-73e628ba550c-openshift-service-ca\") pod \"0375c51e-61a9-420d-9bdf-73e628ba550c\" (UID: \"0375c51e-61a9-420d-9bdf-73e628ba550c\") "
Jul 12 15:47:59.443770 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.443743    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0375c51e-61a9-420d-9bdf-73e628ba550c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0375c51e-61a9-420d-9bdf-73e628ba550c" (UID: "0375c51e-61a9-420d-9bdf-73e628ba550c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:47:59.445603 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.445577    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0375c51e-61a9-420d-9bdf-73e628ba550c" (UID: "0375c51e-61a9-420d-9bdf-73e628ba550c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:47:59.445705 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.445631    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0375c51e-61a9-420d-9bdf-73e628ba550c-kube-api-access-qw9nt" (OuterVolumeSpecName: "kube-api-access-qw9nt") pod "0375c51e-61a9-420d-9bdf-73e628ba550c" (UID: "0375c51e-61a9-420d-9bdf-73e628ba550c"). InnerVolumeSpecName "kube-api-access-qw9nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:47:59.463021 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.462992    2578 generic.go:356] "Generic (PLEG): container finished" podID="0375c51e-61a9-420d-9bdf-73e628ba550c" containerID="53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af" exitCode=0
Jul 12 15:47:59.463157 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.463036    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"
Jul 12 15:47:59.463157 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.463070    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47" event={"ID":"0375c51e-61a9-420d-9bdf-73e628ba550c","Type":"ContainerDied","Data":"53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af"}
Jul 12 15:47:59.463157 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.463131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47" event={"ID":"0375c51e-61a9-420d-9bdf-73e628ba550c","Type":"ContainerDied","Data":"41beeb600087fd9184a3645a34f27397eb566dabeb834120f55ae2fefbd523a5"}
Jul 12 15:47:59.463314 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.463162    2578 scope.go:117] "RemoveContainer" containerID="53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af"
Jul 12 15:47:59.472388 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.472371    2578 scope.go:117] "RemoveContainer" containerID="53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af"
Jul 12 15:47:59.472678 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:47:59.472646    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af\": container with ID starting with 53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af not found: ID does not exist" containerID="53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af"
Jul 12 15:47:59.472732 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.472685    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af"} err="failed to get container status \"53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af\": rpc error: code = NotFound desc = could not find container \"53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af\": container with ID starting with 53af426bc6ffef40e566952d38316e92be5fba8869baa73cb6eba4e1d5a126af not found: ID does not exist"
Jul 12 15:47:59.484691 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.484664    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"]
Jul 12 15:47:59.486379 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.486360    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-c20960-5d9459649-6vl47"]
Jul 12 15:47:59.544618 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.544590    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0375c51e-61a9-420d-9bdf-73e628ba550c-openshift-service-ca\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:47:59.544721 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.544622    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw9nt\" (UniqueName: \"kubernetes.io/projected/0375c51e-61a9-420d-9bdf-73e628ba550c-kube-api-access-qw9nt\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:47:59.544721 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:47:59.544637    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0375c51e-61a9-420d-9bdf-73e628ba550c-maas-api-tls\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:48:00.155462 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:00.155427    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0375c51e-61a9-420d-9bdf-73e628ba550c" path="/var/lib/kubelet/pods/0375c51e-61a9-420d-9bdf-73e628ba550c/volumes"
Jul 12 15:48:08.882042 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:08.882008    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"]
Jul 12 15:48:08.882422 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:08.882396    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0375c51e-61a9-420d-9bdf-73e628ba550c" containerName="maas-api"
Jul 12 15:48:08.882422 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:08.882408    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0375c51e-61a9-420d-9bdf-73e628ba550c" containerName="maas-api"
Jul 12 15:48:08.882494 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:08.882465    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0375c51e-61a9-420d-9bdf-73e628ba550c" containerName="maas-api"
Jul 12 15:48:08.886693 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:08.886676    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:08.889313 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:08.889283    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-b1632a-serving-cert\""
Jul 12 15:48:08.889313 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:08.889288    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-lcn4n\""
Jul 12 15:48:08.894941 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:08.894909    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"]
Jul 12 15:48:09.033313 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.033269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dvc8\" (UniqueName: \"kubernetes.io/projected/46ff1c33-674f-4a9b-bbe9-960ba247c049-kube-api-access-7dvc8\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.033526 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.033421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46ff1c33-674f-4a9b-bbe9-960ba247c049-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.033526 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.033492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.134180 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.134083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dvc8\" (UniqueName: \"kubernetes.io/projected/46ff1c33-674f-4a9b-bbe9-960ba247c049-kube-api-access-7dvc8\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.134180 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.134169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46ff1c33-674f-4a9b-bbe9-960ba247c049-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.134370 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.134214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.134370 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:48:09.134311    2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-b1632a-serving-cert: secret "maas-api-e2e-shared-a-b1632a-serving-cert" not found
Jul 12 15:48:09.134441 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:48:09.134390    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls podName:46ff1c33-674f-4a9b-bbe9-960ba247c049 nodeName:}" failed. No retries permitted until 2026-07-12 15:48:09.634373887 +0000 UTC m=+2308.001974712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls") pod "maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb" (UID: "46ff1c33-674f-4a9b-bbe9-960ba247c049") : secret "maas-api-e2e-shared-a-b1632a-serving-cert" not found
Jul 12 15:48:09.134831 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.134788    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46ff1c33-674f-4a9b-bbe9-960ba247c049-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.142352 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.142321    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dvc8\" (UniqueName: \"kubernetes.io/projected/46ff1c33-674f-4a9b-bbe9-960ba247c049-kube-api-access-7dvc8\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.639526 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.639485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.642065 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.642042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls\") pod \"maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") " pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.799229 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.799183    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:09.936794 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:09.936767    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"]
Jul 12 15:48:09.939421 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:48:09.939394    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ff1c33_674f_4a9b_bbe9_960ba247c049.slice/crio-c8c5d69e03d8fa4b905d62c238e7328c9fd1208d01ce444689fce205960ea8b8 WatchSource:0}: Error finding container c8c5d69e03d8fa4b905d62c238e7328c9fd1208d01ce444689fce205960ea8b8: Status 404 returned error can't find the container with id c8c5d69e03d8fa4b905d62c238e7328c9fd1208d01ce444689fce205960ea8b8
Jul 12 15:48:10.518682 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:10.518594    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb" event={"ID":"46ff1c33-674f-4a9b-bbe9-960ba247c049","Type":"ContainerStarted","Data":"5fb38974c47eb9b833be4c0780e849861e16fb6c99e0b4215ae22e033f2657f3"}
Jul 12 15:48:10.518682 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:10.518652    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb" event={"ID":"46ff1c33-674f-4a9b-bbe9-960ba247c049","Type":"ContainerStarted","Data":"c8c5d69e03d8fa4b905d62c238e7328c9fd1208d01ce444689fce205960ea8b8"}
Jul 12 15:48:10.518869 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:10.518768    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:10.538935 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:10.538883    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb" podStartSLOduration=2.281274571 podStartE2EDuration="2.53886667s" totalImagesPullingTime="257.592099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:48:10.53797895 +0000 UTC m=+2308.905579798" watchObservedRunningTime="2026-07-12 15:48:10.53886667 +0000 UTC m=+2308.906467516"
Jul 12 15:48:16.414861 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.414823    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"]
Jul 12 15:48:16.421823 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.421787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:16.424364 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.424342    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-b08e9f-serving-cert\""
Jul 12 15:48:16.428505 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.428479    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"]
Jul 12 15:48:16.505710 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.505674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:16.505924 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.505721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:16.505924 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.505749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-866sq\" (UniqueName: \"kubernetes.io/projected/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-kube-api-access-866sq\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:16.528607 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.528574    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:48:16.606639 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.606601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:16.606913 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.606665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:16.606913 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.606703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-866sq\" (UniqueName: \"kubernetes.io/projected/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-kube-api-access-866sq\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:16.606913 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:48:16.606878    2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-b08e9f-serving-cert: secret "maas-api-e2e-shared-b-b08e9f-serving-cert" not found
Jul 12 15:48:16.607111 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:48:16.606952    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls podName:33c664cb-5a40-4ba1-ade5-5efe0dd8331d nodeName:}" failed. No retries permitted until 2026-07-12 15:48:17.106935099 +0000 UTC m=+2315.474535927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls") pod "maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx" (UID: "33c664cb-5a40-4ba1-ade5-5efe0dd8331d") : secret "maas-api-e2e-shared-b-b08e9f-serving-cert" not found
Jul 12 15:48:16.607347 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.607331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:16.617286 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:16.617260    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-866sq\" (UniqueName: \"kubernetes.io/projected/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-kube-api-access-866sq\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:17.111505 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:17.111464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:17.114026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:17.114001    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls\") pod \"maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") " pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:17.333509 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:17.333472    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:17.471963 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:17.471926    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"]
Jul 12 15:48:17.475149 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:48:17.475122    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c664cb_5a40_4ba1_ade5_5efe0dd8331d.slice/crio-0280977aa1d9412cfe12aa909005bebacf8d8e893c93b546742c96a274ff7aab WatchSource:0}: Error finding container 0280977aa1d9412cfe12aa909005bebacf8d8e893c93b546742c96a274ff7aab: Status 404 returned error can't find the container with id 0280977aa1d9412cfe12aa909005bebacf8d8e893c93b546742c96a274ff7aab
Jul 12 15:48:17.552422 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:17.552380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx" event={"ID":"33c664cb-5a40-4ba1-ade5-5efe0dd8331d","Type":"ContainerStarted","Data":"0280977aa1d9412cfe12aa909005bebacf8d8e893c93b546742c96a274ff7aab"}
Jul 12 15:48:18.558594 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:18.558547    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx" event={"ID":"33c664cb-5a40-4ba1-ade5-5efe0dd8331d","Type":"ContainerStarted","Data":"6af5ec23618fa5dd5c35d5b978009b8c041c4e071320344865df043d359a4f64"}
Jul 12 15:48:18.559155 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:18.558635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:18.575254 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:18.575203    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx" podStartSLOduration=2.25083116 podStartE2EDuration="2.575187628s" totalImagesPullingTime="324.356468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:48:18.57331303 +0000 UTC m=+2316.940913877" watchObservedRunningTime="2026-07-12 15:48:18.575187628 +0000 UTC m=+2316.942788536"
Jul 12 15:48:24.569491 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:24.569454    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:48:31.243083 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.243049    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"]
Jul 12 15:48:31.246678 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.246656    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.250185 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.250158    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b1632a\"/\"kube-root-ca.crt\""
Jul 12 15:48:31.250185 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.250179    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b1632a\"/\"auth-test-model-b1632a-kserve-self-signed-certs\""
Jul 12 15:48:31.250363 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.250158    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b1632a\"/\"default-dockercfg-q5g8z\""
Jul 12 15:48:31.250363 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.250216    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b1632a\"/\"openshift-service-ca.crt\""
Jul 12 15:48:31.256502 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.256477    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"]
Jul 12 15:48:31.345106 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.345065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd8vx\" (UniqueName: \"kubernetes.io/projected/a18d3ecc-049a-440f-bb26-de21c86bcd8b-kube-api-access-vd8vx\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.345295 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.345123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-model-cache\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.345295 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.345151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-dshm\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.345295 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.345196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tmp-dir\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.345295 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.345246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-home\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.345295 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.345275    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tls-certs\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446332 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd8vx\" (UniqueName: \"kubernetes.io/projected/a18d3ecc-049a-440f-bb26-de21c86bcd8b-kube-api-access-vd8vx\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446332 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-model-cache\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446474    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-dshm\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tmp-dir\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-home\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tls-certs\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446859 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-model-cache\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446889    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tmp-dir\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.446917 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.446905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-home\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.448787 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.448761    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-dshm\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.449009 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.448992    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tls-certs\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.453534 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.453514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd8vx\" (UniqueName: \"kubernetes.io/projected/a18d3ecc-049a-440f-bb26-de21c86bcd8b-kube-api-access-vd8vx\") pod \"auth-test-model-b1632a-kserve-597cfc84ff-fmc88\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") " pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.557555 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.557462    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:31.691364 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:31.691340    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"]
Jul 12 15:48:31.693634 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:48:31.693608    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda18d3ecc_049a_440f_bb26_de21c86bcd8b.slice/crio-898bd3dbff1f35db88c86f71171253a5a89460919539d6d13ea24e57780c56cb WatchSource:0}: Error finding container 898bd3dbff1f35db88c86f71171253a5a89460919539d6d13ea24e57780c56cb: Status 404 returned error can't find the container with id 898bd3dbff1f35db88c86f71171253a5a89460919539d6d13ea24e57780c56cb
Jul 12 15:48:32.629657 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:32.629620    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88" event={"ID":"a18d3ecc-049a-440f-bb26-de21c86bcd8b","Type":"ContainerStarted","Data":"4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509"}
Jul 12 15:48:32.629657 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:32.629661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88" event={"ID":"a18d3ecc-049a-440f-bb26-de21c86bcd8b","Type":"ContainerStarted","Data":"898bd3dbff1f35db88c86f71171253a5a89460919539d6d13ea24e57780c56cb"}
Jul 12 15:48:32.647164 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:32.647111    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88" podStartSLOduration=1.647094493 podStartE2EDuration="1.647094493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:48:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:48:32.64415199 +0000 UTC m=+2331.011752837" watchObservedRunningTime="2026-07-12 15:48:32.647094493 +0000 UTC m=+2331.014695359"
Jul 12 15:48:41.557779 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:41.557688    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:41.557779 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:41.557731    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:41.569775 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:41.569747    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:41.690378 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:41.690346    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:48:50.903818 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:50.903772    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"]
Jul 12 15:48:50.907599 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:50.907583    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:50.911402 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:50.911382    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"openshift-service-ca.crt\""
Jul 12 15:48:50.911502 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:50.911443    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"auth-test-model-b08e9f-kserve-self-signed-certs\""
Jul 12 15:48:50.911502 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:50.911453    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"default-dockercfg-t8rbk\""
Jul 12 15:48:50.911589 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:50.911383    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"kube-root-ca.crt\""
Jul 12 15:48:50.915821 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:50.915759    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"]
Jul 12 15:48:51.042177 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.042143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-home\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.042356 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.042191    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-tmp-dir\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.042356 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.042212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e47b665b-5e8c-4e5b-994b-f801b136ff07-tls-certs\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.042356 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.042280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-dshm\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.042356 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.042323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26ljq\" (UniqueName: \"kubernetes.io/projected/e47b665b-5e8c-4e5b-994b-f801b136ff07-kube-api-access-26ljq\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.042356 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.042346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-model-cache\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.143511 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.143464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-home\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.143693 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.143523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-tmp-dir\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.143693 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.143541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e47b665b-5e8c-4e5b-994b-f801b136ff07-tls-certs\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.143693 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.143564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-dshm\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.143693 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.143606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26ljq\" (UniqueName: \"kubernetes.io/projected/e47b665b-5e8c-4e5b-994b-f801b136ff07-kube-api-access-26ljq\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.143693 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.143625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-model-cache\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.144037 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.144010    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-home\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.144087 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.144024    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-tmp-dir\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.144128 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.144079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-model-cache\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.146072 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.146039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-dshm\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.146300 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.146286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e47b665b-5e8c-4e5b-994b-f801b136ff07-tls-certs\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.151199 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.151177    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26ljq\" (UniqueName: \"kubernetes.io/projected/e47b665b-5e8c-4e5b-994b-f801b136ff07-kube-api-access-26ljq\") pod \"auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") " pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.218682 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.218568    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:48:51.355113 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.354954    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"]
Jul 12 15:48:51.357831 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:48:51.357788    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode47b665b_5e8c_4e5b_994b_f801b136ff07.slice/crio-f053e1709a4c163833aa030f4a747fb94e2a332d56240b985b7c0ad3ae0537c1 WatchSource:0}: Error finding container f053e1709a4c163833aa030f4a747fb94e2a332d56240b985b7c0ad3ae0537c1: Status 404 returned error can't find the container with id f053e1709a4c163833aa030f4a747fb94e2a332d56240b985b7c0ad3ae0537c1
Jul 12 15:48:51.727139 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.727105    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5" event={"ID":"e47b665b-5e8c-4e5b-994b-f801b136ff07","Type":"ContainerStarted","Data":"905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb"}
Jul 12 15:48:51.727323 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.727149    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5" event={"ID":"e47b665b-5e8c-4e5b-994b-f801b136ff07","Type":"ContainerStarted","Data":"f053e1709a4c163833aa030f4a747fb94e2a332d56240b985b7c0ad3ae0537c1"}
Jul 12 15:48:51.743182 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:48:51.743128    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5" podStartSLOduration=1.74310941 podStartE2EDuration="1.74310941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:48:51.741506047 +0000 UTC m=+2350.109106930" watchObservedRunningTime="2026-07-12 15:48:51.74310941 +0000 UTC m=+2350.110710257"
Jul 12 15:49:01.219764 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:49:01.219711    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:49:01.219764 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:49:01.219773    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:49:01.232277 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:49:01.232247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:49:01.786356 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:49:01.786327    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:50:51.535501 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:51.535462    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"]
Jul 12 15:50:52.601026 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:52.600992    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"]
Jul 12 15:50:56.616173 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.616132    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"]
Jul 12 15:50:56.619503 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.619488    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.622007 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.621980    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b1632a\"/\"sub-test-model-b1632a-kserve-self-signed-certs\""
Jul 12 15:50:56.626592 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.626567    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"]
Jul 12 15:50:56.746267 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.746229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tls-certs\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.746471 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.746290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2fdz\" (UniqueName: \"kubernetes.io/projected/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-kube-api-access-h2fdz\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.746471 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.746342    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-model-cache\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.746471 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.746373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-dshm\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.746643 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.746461    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tmp-dir\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.746643 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.746553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-home\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.847918 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.847877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tmp-dir\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.848124 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.847965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-home\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.848124 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.847987    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tls-certs\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.848124 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.848030    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2fdz\" (UniqueName: \"kubernetes.io/projected/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-kube-api-access-h2fdz\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.848124 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.848072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-model-cache\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.848124 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.848114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-dshm\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.848396 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.848335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tmp-dir\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.848457 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.848392    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-home\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.848540 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.848508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-model-cache\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.850379 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.850351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-dshm\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.850552 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.850533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tls-certs\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.857170 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.857146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2fdz\" (UniqueName: \"kubernetes.io/projected/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-kube-api-access-h2fdz\") pod \"sub-test-model-b1632a-kserve-66bd6b599c-m4s4r\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") " pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:56.930200 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:56.930112    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:50:57.064658 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:57.064626    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"]
Jul 12 15:50:57.067082 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:50:57.067054    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0894eb66_6d44_4abb_9c2c_e4a3d3fe7e05.slice/crio-1b4caa5ae9ee70f1ddcc5df71639bae6fa451098a3b70ddb0800c48aec4ecc64 WatchSource:0}: Error finding container 1b4caa5ae9ee70f1ddcc5df71639bae6fa451098a3b70ddb0800c48aec4ecc64: Status 404 returned error can't find the container with id 1b4caa5ae9ee70f1ddcc5df71639bae6fa451098a3b70ddb0800c48aec4ecc64
Jul 12 15:50:57.347374 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:57.347333    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r" event={"ID":"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05","Type":"ContainerStarted","Data":"ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d"}
Jul 12 15:50:57.347639 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:57.347618    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r" event={"ID":"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05","Type":"ContainerStarted","Data":"1b4caa5ae9ee70f1ddcc5df71639bae6fa451098a3b70ddb0800c48aec4ecc64"}
Jul 12 15:50:57.363460 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:50:57.363402    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r" podStartSLOduration=1.363387809 podStartE2EDuration="1.363387809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:50:57.363032116 +0000 UTC m=+2475.730632964" watchObservedRunningTime="2026-07-12 15:50:57.363387809 +0000 UTC m=+2475.730988657"
Jul 12 15:51:06.556339 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.556272    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88" podUID="a18d3ecc-049a-440f-bb26-de21c86bcd8b" containerName="main" containerID="cri-o://4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509" gracePeriod=45
Jul 12 15:51:06.808989 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.808917    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:51:06.847465 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.847427    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vd8vx\" (UniqueName: \"kubernetes.io/projected/a18d3ecc-049a-440f-bb26-de21c86bcd8b-kube-api-access-vd8vx\") pod \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") "
Jul 12 15:51:06.847654 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.847525    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-home\") pod \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") "
Jul 12 15:51:06.847654 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.847563    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tmp-dir\") pod \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") "
Jul 12 15:51:06.847654 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.847606    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tls-certs\") pod \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") "
Jul 12 15:51:06.847837 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.847661    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-dshm\") pod \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") "
Jul 12 15:51:06.847837 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.847690    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-model-cache\") pod \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\" (UID: \"a18d3ecc-049a-440f-bb26-de21c86bcd8b\") "
Jul 12 15:51:06.847837 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.847785    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-home" (OuterVolumeSpecName: "home") pod "a18d3ecc-049a-440f-bb26-de21c86bcd8b" (UID: "a18d3ecc-049a-440f-bb26-de21c86bcd8b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:51:06.847988 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.847931    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a18d3ecc-049a-440f-bb26-de21c86bcd8b" (UID: "a18d3ecc-049a-440f-bb26-de21c86bcd8b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:51:06.848046 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.848016    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a18d3ecc-049a-440f-bb26-de21c86bcd8b" (UID: "a18d3ecc-049a-440f-bb26-de21c86bcd8b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:51:06.848141 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.848121    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:06.848193 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.848147    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:06.849768 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.849730    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18d3ecc-049a-440f-bb26-de21c86bcd8b-kube-api-access-vd8vx" (OuterVolumeSpecName: "kube-api-access-vd8vx") pod "a18d3ecc-049a-440f-bb26-de21c86bcd8b" (UID: "a18d3ecc-049a-440f-bb26-de21c86bcd8b"). InnerVolumeSpecName "kube-api-access-vd8vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:51:06.849768 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.849733    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-dshm" (OuterVolumeSpecName: "dshm") pod "a18d3ecc-049a-440f-bb26-de21c86bcd8b" (UID: "a18d3ecc-049a-440f-bb26-de21c86bcd8b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:51:06.849959 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.849874    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a18d3ecc-049a-440f-bb26-de21c86bcd8b" (UID: "a18d3ecc-049a-440f-bb26-de21c86bcd8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:51:06.930569 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.930528    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:51:06.930771 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.930584    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:51:06.943065 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.943036    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:51:06.948754 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.948733    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a18d3ecc-049a-440f-bb26-de21c86bcd8b-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:06.948754 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.948752    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:06.948754 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.948760    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a18d3ecc-049a-440f-bb26-de21c86bcd8b-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:06.948957 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:06.948767    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd8vx\" (UniqueName: \"kubernetes.io/projected/a18d3ecc-049a-440f-bb26-de21c86bcd8b-kube-api-access-vd8vx\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:07.410193 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.410158    2578 generic.go:356] "Generic (PLEG): container finished" podID="a18d3ecc-049a-440f-bb26-de21c86bcd8b" containerID="4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509" exitCode=0
Jul 12 15:51:07.410377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.410208    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"
Jul 12 15:51:07.410377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.410250    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88" event={"ID":"a18d3ecc-049a-440f-bb26-de21c86bcd8b","Type":"ContainerDied","Data":"4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509"}
Jul 12 15:51:07.410377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.410300    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88" event={"ID":"a18d3ecc-049a-440f-bb26-de21c86bcd8b","Type":"ContainerDied","Data":"898bd3dbff1f35db88c86f71171253a5a89460919539d6d13ea24e57780c56cb"}
Jul 12 15:51:07.410377 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.410318    2578 scope.go:117] "RemoveContainer" containerID="4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509"
Jul 12 15:51:07.421015 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.420991    2578 scope.go:117] "RemoveContainer" containerID="4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509"
Jul 12 15:51:07.421364 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:51:07.421326    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509\": container with ID starting with 4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509 not found: ID does not exist" containerID="4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509"
Jul 12 15:51:07.421422 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.421373    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509"} err="failed to get container status \"4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509\": rpc error: code = NotFound desc = could not find container \"4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509\": container with ID starting with 4f8484d0a377090d754b636873685548e3dc440f66f6d6c43765df5b95053509 not found: ID does not exist"
Jul 12 15:51:07.425407 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.425383    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:51:07.433990 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.433967    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"]
Jul 12 15:51:07.439200 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.439176    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/auth-test-model-b1632a-kserve-597cfc84ff-fmc88"]
Jul 12 15:51:07.621998 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.621955    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5" podUID="e47b665b-5e8c-4e5b-994b-f801b136ff07" containerName="main" containerID="cri-o://905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb" gracePeriod=45
Jul 12 15:51:07.645455 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:51:07.645429    2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode47b665b_5e8c_4e5b_994b_f801b136ff07.slice/crio-conmon-905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb.scope\": RecentStats: unable to find data in memory cache]"
Jul 12 15:51:07.645567 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:51:07.645486    2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode47b665b_5e8c_4e5b_994b_f801b136ff07.slice/crio-905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb.scope\": RecentStats: unable to find data in memory cache]"
Jul 12 15:51:07.882876 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.882847    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:51:07.956294 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956258    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-home\") pod \"e47b665b-5e8c-4e5b-994b-f801b136ff07\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") "
Jul 12 15:51:07.956498 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-model-cache\") pod \"e47b665b-5e8c-4e5b-994b-f801b136ff07\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") "
Jul 12 15:51:07.956498 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956343    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-dshm\") pod \"e47b665b-5e8c-4e5b-994b-f801b136ff07\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") "
Jul 12 15:51:07.956498 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956387    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-tmp-dir\") pod \"e47b665b-5e8c-4e5b-994b-f801b136ff07\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") "
Jul 12 15:51:07.956498 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956419    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e47b665b-5e8c-4e5b-994b-f801b136ff07-tls-certs\") pod \"e47b665b-5e8c-4e5b-994b-f801b136ff07\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") "
Jul 12 15:51:07.956498 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956447    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26ljq\" (UniqueName: \"kubernetes.io/projected/e47b665b-5e8c-4e5b-994b-f801b136ff07-kube-api-access-26ljq\") pod \"e47b665b-5e8c-4e5b-994b-f801b136ff07\" (UID: \"e47b665b-5e8c-4e5b-994b-f801b136ff07\") "
Jul 12 15:51:07.956762 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956585    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-home" (OuterVolumeSpecName: "home") pod "e47b665b-5e8c-4e5b-994b-f801b136ff07" (UID: "e47b665b-5e8c-4e5b-994b-f801b136ff07"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:51:07.956762 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956609    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-model-cache" (OuterVolumeSpecName: "model-cache") pod "e47b665b-5e8c-4e5b-994b-f801b136ff07" (UID: "e47b665b-5e8c-4e5b-994b-f801b136ff07"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:51:07.956762 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956716    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e47b665b-5e8c-4e5b-994b-f801b136ff07" (UID: "e47b665b-5e8c-4e5b-994b-f801b136ff07"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:51:07.956913 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956848    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:07.956913 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956866    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:07.956913 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.956879    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:07.958749 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.958718    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-dshm" (OuterVolumeSpecName: "dshm") pod "e47b665b-5e8c-4e5b-994b-f801b136ff07" (UID: "e47b665b-5e8c-4e5b-994b-f801b136ff07"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:51:07.958895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.958781    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e47b665b-5e8c-4e5b-994b-f801b136ff07-kube-api-access-26ljq" (OuterVolumeSpecName: "kube-api-access-26ljq") pod "e47b665b-5e8c-4e5b-994b-f801b136ff07" (UID: "e47b665b-5e8c-4e5b-994b-f801b136ff07"). InnerVolumeSpecName "kube-api-access-26ljq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:51:07.958895 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:07.958780    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e47b665b-5e8c-4e5b-994b-f801b136ff07-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e47b665b-5e8c-4e5b-994b-f801b136ff07" (UID: "e47b665b-5e8c-4e5b-994b-f801b136ff07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:51:08.057967 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.057872    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e47b665b-5e8c-4e5b-994b-f801b136ff07-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:08.057967 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.057904    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e47b665b-5e8c-4e5b-994b-f801b136ff07-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:08.057967 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.057918    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26ljq\" (UniqueName: \"kubernetes.io/projected/e47b665b-5e8c-4e5b-994b-f801b136ff07-kube-api-access-26ljq\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:51:08.155602 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.155567    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a18d3ecc-049a-440f-bb26-de21c86bcd8b" path="/var/lib/kubelet/pods/a18d3ecc-049a-440f-bb26-de21c86bcd8b/volumes"
Jul 12 15:51:08.416956 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.416920    2578 generic.go:356] "Generic (PLEG): container finished" podID="e47b665b-5e8c-4e5b-994b-f801b136ff07" containerID="905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb" exitCode=0
Jul 12 15:51:08.417132 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.416978    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"
Jul 12 15:51:08.417132 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.417009    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5" event={"ID":"e47b665b-5e8c-4e5b-994b-f801b136ff07","Type":"ContainerDied","Data":"905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb"}
Jul 12 15:51:08.417132 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.417061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5" event={"ID":"e47b665b-5e8c-4e5b-994b-f801b136ff07","Type":"ContainerDied","Data":"f053e1709a4c163833aa030f4a747fb94e2a332d56240b985b7c0ad3ae0537c1"}
Jul 12 15:51:08.417132 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.417083    2578 scope.go:117] "RemoveContainer" containerID="905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb"
Jul 12 15:51:08.427852 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.427829    2578 scope.go:117] "RemoveContainer" containerID="905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb"
Jul 12 15:51:08.428169 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:51:08.428145    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb\": container with ID starting with 905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb not found: ID does not exist" containerID="905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb"
Jul 12 15:51:08.428272 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.428178    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb"} err="failed to get container status \"905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb\": rpc error: code = NotFound desc = could not find container \"905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb\": container with ID starting with 905cb8f7dc660d210d236970447e1b255ea6b2d5c027842f1d809dd3e06bbddb not found: ID does not exist"
Jul 12 15:51:08.433060 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.433036    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"]
Jul 12 15:51:08.438444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:08.438423    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/auth-test-model-b08e9f-kserve-8c45fc68c-hwqc5"]
Jul 12 15:51:10.155783 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:10.155750    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e47b665b-5e8c-4e5b-994b-f801b136ff07" path="/var/lib/kubelet/pods/e47b665b-5e8c-4e5b-994b-f801b136ff07/volumes"
Jul 12 15:51:15.424431 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.424392    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"]
Jul 12 15:51:15.424877 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.424860    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e47b665b-5e8c-4e5b-994b-f801b136ff07" containerName="main"
Jul 12 15:51:15.424931 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.424879    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e47b665b-5e8c-4e5b-994b-f801b136ff07" containerName="main"
Jul 12 15:51:15.424931 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.424910    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18d3ecc-049a-440f-bb26-de21c86bcd8b" containerName="main"
Jul 12 15:51:15.424931 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.424916    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18d3ecc-049a-440f-bb26-de21c86bcd8b" containerName="main"
Jul 12 15:51:15.425025 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.424977    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18d3ecc-049a-440f-bb26-de21c86bcd8b" containerName="main"
Jul 12 15:51:15.425025 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.424987    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e47b665b-5e8c-4e5b-994b-f801b136ff07" containerName="main"
Jul 12 15:51:15.429572 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.429551    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.432321 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.432298    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"kube-root-ca.crt\""
Jul 12 15:51:15.433444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.433423    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"default-dockercfg-t8rbk\""
Jul 12 15:51:15.433605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.433505    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"openshift-service-ca.crt\""
Jul 12 15:51:15.433605 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.433518    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"sub-test-model-b08e9f-kserve-self-signed-certs\""
Jul 12 15:51:15.438535 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.438508    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"]
Jul 12 15:51:15.528973 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.528932    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-model-cache\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.528973 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.528973    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7njwc\" (UniqueName: \"kubernetes.io/projected/565a744e-fd53-436a-a801-d3da36d5382d-kube-api-access-7njwc\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.529177 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.529046    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-dshm\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.529177 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.529071    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-home\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.529177 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.529137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-tmp-dir\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.529177 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.529170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/565a744e-fd53-436a-a801-d3da36d5382d-tls-certs\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630035 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.629991    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-model-cache\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630035 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.630037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7njwc\" (UniqueName: \"kubernetes.io/projected/565a744e-fd53-436a-a801-d3da36d5382d-kube-api-access-7njwc\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630267 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.630094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-dshm\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630267 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.630120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-home\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630267 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.630161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-tmp-dir\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630267 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.630184    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/565a744e-fd53-436a-a801-d3da36d5382d-tls-certs\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630501 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.630477    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-model-cache\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630501 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.630491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-home\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.630570 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.630537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-tmp-dir\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.632591 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.632555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-dshm\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.632708 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.632647    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/565a744e-fd53-436a-a801-d3da36d5382d-tls-certs\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.639672 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.639645    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7njwc\" (UniqueName: \"kubernetes.io/projected/565a744e-fd53-436a-a801-d3da36d5382d-kube-api-access-7njwc\") pod \"sub-test-model-b08e9f-kserve-795994b6cf-pdmff\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") " pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.739662 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.739558    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:15.875219 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:15.875183    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"]
Jul 12 15:51:15.878343 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:51:15.878309    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod565a744e_fd53_436a_a801_d3da36d5382d.slice/crio-460e1c629a7c0156b7aba84b0d82fb19d76fc0f382add225eea3cdecebcbb217 WatchSource:0}: Error finding container 460e1c629a7c0156b7aba84b0d82fb19d76fc0f382add225eea3cdecebcbb217: Status 404 returned error can't find the container with id 460e1c629a7c0156b7aba84b0d82fb19d76fc0f382add225eea3cdecebcbb217
Jul 12 15:51:16.463837 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:16.463767    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff" event={"ID":"565a744e-fd53-436a-a801-d3da36d5382d","Type":"ContainerStarted","Data":"fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248"}
Jul 12 15:51:16.463837 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:16.463831    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff" event={"ID":"565a744e-fd53-436a-a801-d3da36d5382d","Type":"ContainerStarted","Data":"460e1c629a7c0156b7aba84b0d82fb19d76fc0f382add225eea3cdecebcbb217"}
Jul 12 15:51:16.479332 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:16.479277    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff" podStartSLOduration=1.4792607 podStartE2EDuration="1.4792607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:51:16.478686172 +0000 UTC m=+2494.846287019" watchObservedRunningTime="2026-07-12 15:51:16.4792607 +0000 UTC m=+2494.846861547"
Jul 12 15:51:25.740455 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:25.740421    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:25.741077 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:25.740491    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:25.753094 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:25.753068    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:26.524657 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:26.524627    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:51:52.594936 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:52.594898    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"]
Jul 12 15:51:53.595073 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:51:53.595040    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"]
Jul 12 15:52:04.614351 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.614315    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"]
Jul 12 15:52:04.618748 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.618722    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.621278 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.621255    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b1632a\"/\"rate-test-model-b1632a-kserve-self-signed-certs\""
Jul 12 15:52:04.626709 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.626362    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"]
Jul 12 15:52:04.719780 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.719743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-home\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.720006 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.719834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-model-cache\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.720006 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.719864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-tmp-dir\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.720006 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.719952    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-dshm\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.720142 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.720031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/686e823e-6933-4a41-8e24-99fb9dbfc274-tls-certs\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.720142 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.720077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scm2t\" (UniqueName: \"kubernetes.io/projected/686e823e-6933-4a41-8e24-99fb9dbfc274-kube-api-access-scm2t\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.821304 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.821269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-dshm\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.821478 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.821341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/686e823e-6933-4a41-8e24-99fb9dbfc274-tls-certs\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.821478 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.821377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scm2t\" (UniqueName: \"kubernetes.io/projected/686e823e-6933-4a41-8e24-99fb9dbfc274-kube-api-access-scm2t\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.821478 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.821413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-home\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.821478 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.821454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-model-cache\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.821666 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.821503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-tmp-dir\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.821940 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.821912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-home\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.822050 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.821958    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-model-cache\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.822050 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.822001    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-tmp-dir\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.823658 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.823638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-dshm\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.823828 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.823792    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/686e823e-6933-4a41-8e24-99fb9dbfc274-tls-certs\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.829234 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.829205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scm2t\" (UniqueName: \"kubernetes.io/projected/686e823e-6933-4a41-8e24-99fb9dbfc274-kube-api-access-scm2t\") pod \"rate-test-model-b1632a-kserve-85bf94788c-sk67s\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") " pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:04.929996 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:04.929907    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:05.071758 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:05.071728    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"]
Jul 12 15:52:05.074463 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:52:05.074435    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod686e823e_6933_4a41_8e24_99fb9dbfc274.slice/crio-f5a4ffea735e0642e1ec4f8b9a456bcf212783bbbedeab5f5731bf8194e802fc WatchSource:0}: Error finding container f5a4ffea735e0642e1ec4f8b9a456bcf212783bbbedeab5f5731bf8194e802fc: Status 404 returned error can't find the container with id f5a4ffea735e0642e1ec4f8b9a456bcf212783bbbedeab5f5731bf8194e802fc
Jul 12 15:52:05.705996 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:05.705963    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s" event={"ID":"686e823e-6933-4a41-8e24-99fb9dbfc274","Type":"ContainerStarted","Data":"f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919"}
Jul 12 15:52:05.705996 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:05.706004    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s" event={"ID":"686e823e-6933-4a41-8e24-99fb9dbfc274","Type":"ContainerStarted","Data":"f5a4ffea735e0642e1ec4f8b9a456bcf212783bbbedeab5f5731bf8194e802fc"}
Jul 12 15:52:05.724918 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:05.724859    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s" podStartSLOduration=1.7248433589999999 podStartE2EDuration="1.724843359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:52:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:52:05.722345898 +0000 UTC m=+2544.089946746" watchObservedRunningTime="2026-07-12 15:52:05.724843359 +0000 UTC m=+2544.092444244"
Jul 12 15:52:07.617037 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:07.616989    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r" podUID="0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" containerName="main" containerID="cri-o://ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d" gracePeriod=45
Jul 12 15:52:07.647972 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:52:07.647926    2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0894eb66_6d44_4abb_9c2c_e4a3d3fe7e05.slice/crio-conmon-ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d.scope\": RecentStats: unable to find data in memory cache]"
Jul 12 15:52:07.896042 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:07.896011    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:52:08.053777 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.053741    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tls-certs\") pod \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") "
Jul 12 15:52:08.053986 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.053864    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-model-cache\") pod \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") "
Jul 12 15:52:08.053986 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.053889    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-home\") pod \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") "
Jul 12 15:52:08.053986 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.053916    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-dshm\") pod \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") "
Jul 12 15:52:08.053986 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.053930    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tmp-dir\") pod \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") "
Jul 12 15:52:08.054204 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.054011    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2fdz\" (UniqueName: \"kubernetes.io/projected/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-kube-api-access-h2fdz\") pod \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\" (UID: \"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05\") "
Jul 12 15:52:08.054204 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.054148    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-model-cache" (OuterVolumeSpecName: "model-cache") pod "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" (UID: "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:52:08.054204 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.054160    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-home" (OuterVolumeSpecName: "home") pod "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" (UID: "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:52:08.054359 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.054240    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" (UID: "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:52:08.054412 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.054393    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.054500 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.054411    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.054500 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.054424    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.056059 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.056033    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" (UID: "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:52:08.056176 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.056115    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-kube-api-access-h2fdz" (OuterVolumeSpecName: "kube-api-access-h2fdz") pod "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" (UID: "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05"). InnerVolumeSpecName "kube-api-access-h2fdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:52:08.056354 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.056334    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-dshm" (OuterVolumeSpecName: "dshm") pod "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" (UID: "0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:52:08.155513 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.155420    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2fdz\" (UniqueName: \"kubernetes.io/projected/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-kube-api-access-h2fdz\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.155513 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.155447    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.155513 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.155457    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.615667 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.615625    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff" podUID="565a744e-fd53-436a-a801-d3da36d5382d" containerName="main" containerID="cri-o://fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248" gracePeriod=45
Jul 12 15:52:08.723405 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.723372    2578 generic.go:356] "Generic (PLEG): container finished" podID="0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" containerID="ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d" exitCode=0
Jul 12 15:52:08.723851 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.723433    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"
Jul 12 15:52:08.723851 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.723450    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r" event={"ID":"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05","Type":"ContainerDied","Data":"ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d"}
Jul 12 15:52:08.723851 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.723495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r" event={"ID":"0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05","Type":"ContainerDied","Data":"1b4caa5ae9ee70f1ddcc5df71639bae6fa451098a3b70ddb0800c48aec4ecc64"}
Jul 12 15:52:08.723851 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.723513    2578 scope.go:117] "RemoveContainer" containerID="ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d"
Jul 12 15:52:08.732892 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.732872    2578 scope.go:117] "RemoveContainer" containerID="ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d"
Jul 12 15:52:08.733248 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:52:08.733227    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d\": container with ID starting with ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d not found: ID does not exist" containerID="ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d"
Jul 12 15:52:08.733311 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.733258    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d"} err="failed to get container status \"ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d\": rpc error: code = NotFound desc = could not find container \"ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d\": container with ID starting with ccb029afede1c1d699292fba8277f83c6eac2517d14ad901b1216124b5e0bb9d not found: ID does not exist"
Jul 12 15:52:08.740534 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.740496    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"]
Jul 12 15:52:08.743114 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.743090    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/sub-test-model-b1632a-kserve-66bd6b599c-m4s4r"]
Jul 12 15:52:08.876462 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.876391    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:52:08.963100 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963061    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-tmp-dir\") pod \"565a744e-fd53-436a-a801-d3da36d5382d\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") "
Jul 12 15:52:08.963306 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963125    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7njwc\" (UniqueName: \"kubernetes.io/projected/565a744e-fd53-436a-a801-d3da36d5382d-kube-api-access-7njwc\") pod \"565a744e-fd53-436a-a801-d3da36d5382d\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") "
Jul 12 15:52:08.963306 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963159    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-model-cache\") pod \"565a744e-fd53-436a-a801-d3da36d5382d\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") "
Jul 12 15:52:08.963306 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963202    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-home\") pod \"565a744e-fd53-436a-a801-d3da36d5382d\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") "
Jul 12 15:52:08.963306 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963243    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-dshm\") pod \"565a744e-fd53-436a-a801-d3da36d5382d\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") "
Jul 12 15:52:08.963527 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963346    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/565a744e-fd53-436a-a801-d3da36d5382d-tls-certs\") pod \"565a744e-fd53-436a-a801-d3da36d5382d\" (UID: \"565a744e-fd53-436a-a801-d3da36d5382d\") "
Jul 12 15:52:08.963527 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963342    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "565a744e-fd53-436a-a801-d3da36d5382d" (UID: "565a744e-fd53-436a-a801-d3da36d5382d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:52:08.963527 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963440    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-model-cache" (OuterVolumeSpecName: "model-cache") pod "565a744e-fd53-436a-a801-d3da36d5382d" (UID: "565a744e-fd53-436a-a801-d3da36d5382d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:52:08.963663 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963525    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-home" (OuterVolumeSpecName: "home") pod "565a744e-fd53-436a-a801-d3da36d5382d" (UID: "565a744e-fd53-436a-a801-d3da36d5382d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:52:08.963729 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963708    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.963766 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963734    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.963766 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.963747    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:08.965461 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.965432    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/565a744e-fd53-436a-a801-d3da36d5382d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "565a744e-fd53-436a-a801-d3da36d5382d" (UID: "565a744e-fd53-436a-a801-d3da36d5382d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:52:08.965557 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.965485    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-dshm" (OuterVolumeSpecName: "dshm") pod "565a744e-fd53-436a-a801-d3da36d5382d" (UID: "565a744e-fd53-436a-a801-d3da36d5382d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:52:08.965557 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:08.965504    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/565a744e-fd53-436a-a801-d3da36d5382d-kube-api-access-7njwc" (OuterVolumeSpecName: "kube-api-access-7njwc") pod "565a744e-fd53-436a-a801-d3da36d5382d" (UID: "565a744e-fd53-436a-a801-d3da36d5382d"). InnerVolumeSpecName "kube-api-access-7njwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:52:09.064992 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.064950    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/565a744e-fd53-436a-a801-d3da36d5382d-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:09.064992 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.064985    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/565a744e-fd53-436a-a801-d3da36d5382d-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:09.064992 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.064997    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7njwc\" (UniqueName: \"kubernetes.io/projected/565a744e-fd53-436a-a801-d3da36d5382d-kube-api-access-7njwc\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:52:09.730260 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.730224    2578 generic.go:356] "Generic (PLEG): container finished" podID="565a744e-fd53-436a-a801-d3da36d5382d" containerID="fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248" exitCode=0
Jul 12 15:52:09.730744 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.730281    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"
Jul 12 15:52:09.730744 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.730305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff" event={"ID":"565a744e-fd53-436a-a801-d3da36d5382d","Type":"ContainerDied","Data":"fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248"}
Jul 12 15:52:09.730744 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.730348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff" event={"ID":"565a744e-fd53-436a-a801-d3da36d5382d","Type":"ContainerDied","Data":"460e1c629a7c0156b7aba84b0d82fb19d76fc0f382add225eea3cdecebcbb217"}
Jul 12 15:52:09.730744 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.730363    2578 scope.go:117] "RemoveContainer" containerID="fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248"
Jul 12 15:52:09.739967 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.739948    2578 scope.go:117] "RemoveContainer" containerID="fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248"
Jul 12 15:52:09.740232 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:52:09.740214    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248\": container with ID starting with fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248 not found: ID does not exist" containerID="fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248"
Jul 12 15:52:09.740291 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.740240    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248"} err="failed to get container status \"fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248\": rpc error: code = NotFound desc = could not find container \"fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248\": container with ID starting with fbf56272b60dbda4bd78ec6906bbb59998803a3e8b68e1616559fce107e57248 not found: ID does not exist"
Jul 12 15:52:09.750718 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.750628    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"]
Jul 12 15:52:09.752506 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:09.752486    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/sub-test-model-b08e9f-kserve-795994b6cf-pdmff"]
Jul 12 15:52:10.156308 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:10.156275    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" path="/var/lib/kubelet/pods/0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05/volumes"
Jul 12 15:52:10.156739 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:10.156725    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="565a744e-fd53-436a-a801-d3da36d5382d" path="/var/lib/kubelet/pods/565a744e-fd53-436a-a801-d3da36d5382d/volumes"
Jul 12 15:52:14.930457 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:14.930419    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:14.930957 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:14.930474    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:14.943234 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:14.943203    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:15.772414 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:15.772376    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:52:26.212389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.212348    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"]
Jul 12 15:52:26.212934 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.212915    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" containerName="main"
Jul 12 15:52:26.213228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.212936    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" containerName="main"
Jul 12 15:52:26.213228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.212947    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565a744e-fd53-436a-a801-d3da36d5382d" containerName="main"
Jul 12 15:52:26.213228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.212952    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="565a744e-fd53-436a-a801-d3da36d5382d" containerName="main"
Jul 12 15:52:26.213228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.213017    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="565a744e-fd53-436a-a801-d3da36d5382d" containerName="main"
Jul 12 15:52:26.213228 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.213024    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0894eb66-6d44-4abb-9c2c-e4a3d3fe7e05" containerName="main"
Jul 12 15:52:26.216160 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.216140    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.219872 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.219844    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"default-dockercfg-t8rbk\""
Jul 12 15:52:26.220030 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.219846    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"openshift-service-ca.crt\""
Jul 12 15:52:26.220030 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.219846    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"rate-test-model-b08e9f-kserve-self-signed-certs\""
Jul 12 15:52:26.220030 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.219852    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b08e9f\"/\"kube-root-ca.crt\""
Jul 12 15:52:26.224660 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.224633    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"]
Jul 12 15:52:26.232191 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.232153    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-home\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.232406 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.232213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-model-cache\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.232406 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.232257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a890e49b-1f52-4ff4-b6e0-42c60223831f-tls-certs\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.232406 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.232291    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-dshm\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.232406 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.232363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-tmp-dir\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.232585 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.232463    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn5fg\" (UniqueName: \"kubernetes.io/projected/a890e49b-1f52-4ff4-b6e0-42c60223831f-kube-api-access-hn5fg\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.333416 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-tmp-dir\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.333601 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn5fg\" (UniqueName: \"kubernetes.io/projected/a890e49b-1f52-4ff4-b6e0-42c60223831f-kube-api-access-hn5fg\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.333601 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333497    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-home\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.333601 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-model-cache\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.333601 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a890e49b-1f52-4ff4-b6e0-42c60223831f-tls-certs\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.333601 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333589    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-dshm\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.333918 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-tmp-dir\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.333976 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-model-cache\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.334037 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.333980    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-home\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.335960 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.335938    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-dshm\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.336220 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.336204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a890e49b-1f52-4ff4-b6e0-42c60223831f-tls-certs\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.341357 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.341334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn5fg\" (UniqueName: \"kubernetes.io/projected/a890e49b-1f52-4ff4-b6e0-42c60223831f-kube-api-access-hn5fg\") pod \"rate-test-model-b08e9f-kserve-f8787756c-xcxgc\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") " pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.527282 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.527184    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:26.663931 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.663901    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"]
Jul 12 15:52:26.665940 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:52:26.665910    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda890e49b_1f52_4ff4_b6e0_42c60223831f.slice/crio-223080a5c137675ef5349322bfcb9699b85a64cf862e19658713983a0028e6c8 WatchSource:0}: Error finding container 223080a5c137675ef5349322bfcb9699b85a64cf862e19658713983a0028e6c8: Status 404 returned error can't find the container with id 223080a5c137675ef5349322bfcb9699b85a64cf862e19658713983a0028e6c8
Jul 12 15:52:26.816462 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.816367    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc" event={"ID":"a890e49b-1f52-4ff4-b6e0-42c60223831f","Type":"ContainerStarted","Data":"468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912"}
Jul 12 15:52:26.816462 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:26.816415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc" event={"ID":"a890e49b-1f52-4ff4-b6e0-42c60223831f","Type":"ContainerStarted","Data":"223080a5c137675ef5349322bfcb9699b85a64cf862e19658713983a0028e6c8"}
Jul 12 15:52:36.527800 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:36.527768    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:36.528311 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:36.527846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:36.540736 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:36.540705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:52:36.557190 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:36.557133    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc" podStartSLOduration=10.557116329 podStartE2EDuration="10.557116329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:52:26.83112703 +0000 UTC m=+2565.198727878" watchObservedRunningTime="2026-07-12 15:52:36.557116329 +0000 UTC m=+2574.924717215"
Jul 12 15:52:36.877978 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:52:36.877949    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:53:09.032604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:09.032512    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"]
Jul 12 15:53:10.011152 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:10.011116    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"]
Jul 12 15:53:11.609875 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:11.609836    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"]
Jul 12 15:53:11.610336 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:11.610115    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb" podUID="46ff1c33-674f-4a9b-bbe9-960ba247c049" containerName="maas-api" containerID="cri-o://5fb38974c47eb9b833be4c0780e849861e16fb6c99e0b4215ae22e033f2657f3" gracePeriod=30
Jul 12 15:53:12.049054 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.049017    2578 generic.go:356] "Generic (PLEG): container finished" podID="46ff1c33-674f-4a9b-bbe9-960ba247c049" containerID="5fb38974c47eb9b833be4c0780e849861e16fb6c99e0b4215ae22e033f2657f3" exitCode=0
Jul 12 15:53:12.049215 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.049092    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb" event={"ID":"46ff1c33-674f-4a9b-bbe9-960ba247c049","Type":"ContainerDied","Data":"5fb38974c47eb9b833be4c0780e849861e16fb6c99e0b4215ae22e033f2657f3"}
Jul 12 15:53:12.049215 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.049148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb" event={"ID":"46ff1c33-674f-4a9b-bbe9-960ba247c049","Type":"ContainerDied","Data":"c8c5d69e03d8fa4b905d62c238e7328c9fd1208d01ce444689fce205960ea8b8"}
Jul 12 15:53:12.049215 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.049165    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8c5d69e03d8fa4b905d62c238e7328c9fd1208d01ce444689fce205960ea8b8"
Jul 12 15:53:12.056078 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.056054    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:53:12.185052 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.184959    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dvc8\" (UniqueName: \"kubernetes.io/projected/46ff1c33-674f-4a9b-bbe9-960ba247c049-kube-api-access-7dvc8\") pod \"46ff1c33-674f-4a9b-bbe9-960ba247c049\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") "
Jul 12 15:53:12.185052 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.185029    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls\") pod \"46ff1c33-674f-4a9b-bbe9-960ba247c049\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") "
Jul 12 15:53:12.185245 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.185130    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46ff1c33-674f-4a9b-bbe9-960ba247c049-openshift-service-ca\") pod \"46ff1c33-674f-4a9b-bbe9-960ba247c049\" (UID: \"46ff1c33-674f-4a9b-bbe9-960ba247c049\") "
Jul 12 15:53:12.185517 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.185487    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46ff1c33-674f-4a9b-bbe9-960ba247c049-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46ff1c33-674f-4a9b-bbe9-960ba247c049" (UID: "46ff1c33-674f-4a9b-bbe9-960ba247c049"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:53:12.187198 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.187174    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46ff1c33-674f-4a9b-bbe9-960ba247c049-kube-api-access-7dvc8" (OuterVolumeSpecName: "kube-api-access-7dvc8") pod "46ff1c33-674f-4a9b-bbe9-960ba247c049" (UID: "46ff1c33-674f-4a9b-bbe9-960ba247c049"). InnerVolumeSpecName "kube-api-access-7dvc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:53:12.187275 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.187245    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46ff1c33-674f-4a9b-bbe9-960ba247c049" (UID: "46ff1c33-674f-4a9b-bbe9-960ba247c049"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:53:12.286690 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.286640    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dvc8\" (UniqueName: \"kubernetes.io/projected/46ff1c33-674f-4a9b-bbe9-960ba247c049-kube-api-access-7dvc8\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:12.286690 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.286680    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46ff1c33-674f-4a9b-bbe9-960ba247c049-maas-api-tls\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:12.286690 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:12.286696    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46ff1c33-674f-4a9b-bbe9-960ba247c049-openshift-service-ca\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:13.054856 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:13.054826    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"
Jul 12 15:53:13.075523 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:13.075484    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"]
Jul 12 15:53:13.078926 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:13.078897    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b1632a-6b9b9cb54b-nvvqb"]
Jul 12 15:53:14.156011 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:14.155977    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46ff1c33-674f-4a9b-bbe9-960ba247c049" path="/var/lib/kubelet/pods/46ff1c33-674f-4a9b-bbe9-960ba247c049/volumes"
Jul 12 15:53:24.052844 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.052786    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s" podUID="686e823e-6933-4a41-8e24-99fb9dbfc274" containerName="main" containerID="cri-o://f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919" gracePeriod=45
Jul 12 15:53:24.311899 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.311829    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:53:24.403741 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.403704    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-tmp-dir\") pod \"686e823e-6933-4a41-8e24-99fb9dbfc274\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") "
Jul 12 15:53:24.403950 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.403769    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-dshm\") pod \"686e823e-6933-4a41-8e24-99fb9dbfc274\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") "
Jul 12 15:53:24.403950 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.403796    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/686e823e-6933-4a41-8e24-99fb9dbfc274-tls-certs\") pod \"686e823e-6933-4a41-8e24-99fb9dbfc274\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") "
Jul 12 15:53:24.403950 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.403853    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-model-cache\") pod \"686e823e-6933-4a41-8e24-99fb9dbfc274\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") "
Jul 12 15:53:24.403950 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.403894    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scm2t\" (UniqueName: \"kubernetes.io/projected/686e823e-6933-4a41-8e24-99fb9dbfc274-kube-api-access-scm2t\") pod \"686e823e-6933-4a41-8e24-99fb9dbfc274\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") "
Jul 12 15:53:24.404177 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.403956    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-home\") pod \"686e823e-6933-4a41-8e24-99fb9dbfc274\" (UID: \"686e823e-6933-4a41-8e24-99fb9dbfc274\") "
Jul 12 15:53:24.404177 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.404066    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "686e823e-6933-4a41-8e24-99fb9dbfc274" (UID: "686e823e-6933-4a41-8e24-99fb9dbfc274"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:53:24.404177 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.404163    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-model-cache" (OuterVolumeSpecName: "model-cache") pod "686e823e-6933-4a41-8e24-99fb9dbfc274" (UID: "686e823e-6933-4a41-8e24-99fb9dbfc274"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:53:24.404318 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.404276    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-home" (OuterVolumeSpecName: "home") pod "686e823e-6933-4a41-8e24-99fb9dbfc274" (UID: "686e823e-6933-4a41-8e24-99fb9dbfc274"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:53:24.404416 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.404392    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:24.404416 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.404415    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:24.404604 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.404424    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:24.406005 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.405978    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/686e823e-6933-4a41-8e24-99fb9dbfc274-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "686e823e-6933-4a41-8e24-99fb9dbfc274" (UID: "686e823e-6933-4a41-8e24-99fb9dbfc274"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:53:24.406092 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.405995    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/686e823e-6933-4a41-8e24-99fb9dbfc274-kube-api-access-scm2t" (OuterVolumeSpecName: "kube-api-access-scm2t") pod "686e823e-6933-4a41-8e24-99fb9dbfc274" (UID: "686e823e-6933-4a41-8e24-99fb9dbfc274"). InnerVolumeSpecName "kube-api-access-scm2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:53:24.406092 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.406038    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-dshm" (OuterVolumeSpecName: "dshm") pod "686e823e-6933-4a41-8e24-99fb9dbfc274" (UID: "686e823e-6933-4a41-8e24-99fb9dbfc274"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:53:24.505499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.505463    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/686e823e-6933-4a41-8e24-99fb9dbfc274-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:24.505499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.505492    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/686e823e-6933-4a41-8e24-99fb9dbfc274-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:24.505499 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:24.505505    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scm2t\" (UniqueName: \"kubernetes.io/projected/686e823e-6933-4a41-8e24-99fb9dbfc274-kube-api-access-scm2t\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:25.031722 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.031676    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc" podUID="a890e49b-1f52-4ff4-b6e0-42c60223831f" containerName="main" containerID="cri-o://468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912" gracePeriod=45
Jul 12 15:53:25.117919 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.117884    2578 generic.go:356] "Generic (PLEG): container finished" podID="686e823e-6933-4a41-8e24-99fb9dbfc274" containerID="f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919" exitCode=0
Jul 12 15:53:25.118358 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.118013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s" event={"ID":"686e823e-6933-4a41-8e24-99fb9dbfc274","Type":"ContainerDied","Data":"f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919"}
Jul 12 15:53:25.118358 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.118019    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"
Jul 12 15:53:25.118358 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.118046    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s" event={"ID":"686e823e-6933-4a41-8e24-99fb9dbfc274","Type":"ContainerDied","Data":"f5a4ffea735e0642e1ec4f8b9a456bcf212783bbbedeab5f5731bf8194e802fc"}
Jul 12 15:53:25.118358 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.118063    2578 scope.go:117] "RemoveContainer" containerID="f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919"
Jul 12 15:53:25.140601 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.140568    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"]
Jul 12 15:53:25.143156 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.143133    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b1632a/rate-test-model-b1632a-kserve-85bf94788c-sk67s"]
Jul 12 15:53:25.143634 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.143615    2578 scope.go:117] "RemoveContainer" containerID="f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919"
Jul 12 15:53:25.143971 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:53:25.143953    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919\": container with ID starting with f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919 not found: ID does not exist" containerID="f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919"
Jul 12 15:53:25.144058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.143982    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919"} err="failed to get container status \"f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919\": rpc error: code = NotFound desc = could not find container \"f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919\": container with ID starting with f248059a05b50f5193022af5b7ee08d4e9af03b30262b502700dc22bb9e84919 not found: ID does not exist"
Jul 12 15:53:25.283546 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.283480    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:53:25.414959 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.414927    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-tmp-dir\") pod \"a890e49b-1f52-4ff4-b6e0-42c60223831f\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") "
Jul 12 15:53:25.414959 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.414966    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-dshm\") pod \"a890e49b-1f52-4ff4-b6e0-42c60223831f\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") "
Jul 12 15:53:25.415226 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-model-cache\") pod \"a890e49b-1f52-4ff4-b6e0-42c60223831f\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") "
Jul 12 15:53:25.415226 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415028    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a890e49b-1f52-4ff4-b6e0-42c60223831f-tls-certs\") pod \"a890e49b-1f52-4ff4-b6e0-42c60223831f\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") "
Jul 12 15:53:25.415226 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415047    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-home\") pod \"a890e49b-1f52-4ff4-b6e0-42c60223831f\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") "
Jul 12 15:53:25.415226 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415064    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn5fg\" (UniqueName: \"kubernetes.io/projected/a890e49b-1f52-4ff4-b6e0-42c60223831f-kube-api-access-hn5fg\") pod \"a890e49b-1f52-4ff4-b6e0-42c60223831f\" (UID: \"a890e49b-1f52-4ff4-b6e0-42c60223831f\") "
Jul 12 15:53:25.415440 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415304    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-model-cache" (OuterVolumeSpecName: "model-cache") pod "a890e49b-1f52-4ff4-b6e0-42c60223831f" (UID: "a890e49b-1f52-4ff4-b6e0-42c60223831f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:53:25.415440 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415298    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a890e49b-1f52-4ff4-b6e0-42c60223831f" (UID: "a890e49b-1f52-4ff4-b6e0-42c60223831f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:53:25.415440 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415384    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-home" (OuterVolumeSpecName: "home") pod "a890e49b-1f52-4ff4-b6e0-42c60223831f" (UID: "a890e49b-1f52-4ff4-b6e0-42c60223831f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:53:25.415440 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415413    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-tmp-dir\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:25.415440 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.415431    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-model-cache\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:25.417278 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.417251    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-dshm" (OuterVolumeSpecName: "dshm") pod "a890e49b-1f52-4ff4-b6e0-42c60223831f" (UID: "a890e49b-1f52-4ff4-b6e0-42c60223831f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 15:53:25.417365 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.417325    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a890e49b-1f52-4ff4-b6e0-42c60223831f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a890e49b-1f52-4ff4-b6e0-42c60223831f" (UID: "a890e49b-1f52-4ff4-b6e0-42c60223831f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:53:25.417365 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.417335    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a890e49b-1f52-4ff4-b6e0-42c60223831f-kube-api-access-hn5fg" (OuterVolumeSpecName: "kube-api-access-hn5fg") pod "a890e49b-1f52-4ff4-b6e0-42c60223831f" (UID: "a890e49b-1f52-4ff4-b6e0-42c60223831f"). InnerVolumeSpecName "kube-api-access-hn5fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:53:25.516883 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.516835    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-dshm\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:25.516883 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.516872    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a890e49b-1f52-4ff4-b6e0-42c60223831f-tls-certs\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:25.516883 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.516888    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a890e49b-1f52-4ff4-b6e0-42c60223831f-home\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:25.517134 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:25.516899    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn5fg\" (UniqueName: \"kubernetes.io/projected/a890e49b-1f52-4ff4-b6e0-42c60223831f-kube-api-access-hn5fg\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:26.125213 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.125178    2578 generic.go:356] "Generic (PLEG): container finished" podID="a890e49b-1f52-4ff4-b6e0-42c60223831f" containerID="468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912" exitCode=0
Jul 12 15:53:26.125630 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.125221    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"
Jul 12 15:53:26.125630 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.125260    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc" event={"ID":"a890e49b-1f52-4ff4-b6e0-42c60223831f","Type":"ContainerDied","Data":"468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912"}
Jul 12 15:53:26.125630 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.125303    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc" event={"ID":"a890e49b-1f52-4ff4-b6e0-42c60223831f","Type":"ContainerDied","Data":"223080a5c137675ef5349322bfcb9699b85a64cf862e19658713983a0028e6c8"}
Jul 12 15:53:26.125630 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.125326    2578 scope.go:117] "RemoveContainer" containerID="468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912"
Jul 12 15:53:26.134977 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.134960    2578 scope.go:117] "RemoveContainer" containerID="468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912"
Jul 12 15:53:26.135261 ip-10-0-142-238 kubenswrapper[2578]: E0712 15:53:26.135243    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912\": container with ID starting with 468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912 not found: ID does not exist" containerID="468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912"
Jul 12 15:53:26.135316 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.135273    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912"} err="failed to get container status \"468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912\": rpc error: code = NotFound desc = could not find container \"468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912\": container with ID starting with 468c8cd696fc4097cc603c7d7b7c5e9a76e503f0c030e891c97e039bdac1e912 not found: ID does not exist"
Jul 12 15:53:26.145560 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.145535    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"]
Jul 12 15:53:26.154930 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.154907    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="686e823e-6933-4a41-8e24-99fb9dbfc274" path="/var/lib/kubelet/pods/686e823e-6933-4a41-8e24-99fb9dbfc274/volumes"
Jul 12 15:53:26.155170 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:26.155157    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b08e9f/rate-test-model-b08e9f-kserve-f8787756c-xcxgc"]
Jul 12 15:53:28.154780 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:28.154733    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a890e49b-1f52-4ff4-b6e0-42c60223831f" path="/var/lib/kubelet/pods/a890e49b-1f52-4ff4-b6e0-42c60223831f/volumes"
Jul 12 15:53:54.104583 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.104549    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"]
Jul 12 15:53:54.105119 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.104768    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx" podUID="33c664cb-5a40-4ba1-ade5-5efe0dd8331d" containerName="maas-api" containerID="cri-o://6af5ec23618fa5dd5c35d5b978009b8c041c4e071320344865df043d359a4f64" gracePeriod=30
Jul 12 15:53:54.272930 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.272897    2578 generic.go:356] "Generic (PLEG): container finished" podID="33c664cb-5a40-4ba1-ade5-5efe0dd8331d" containerID="6af5ec23618fa5dd5c35d5b978009b8c041c4e071320344865df043d359a4f64" exitCode=0
Jul 12 15:53:54.273082 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.272964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx" event={"ID":"33c664cb-5a40-4ba1-ade5-5efe0dd8331d","Type":"ContainerDied","Data":"6af5ec23618fa5dd5c35d5b978009b8c041c4e071320344865df043d359a4f64"}
Jul 12 15:53:54.358304 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.358236    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:53:54.488166 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.488124    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-openshift-service-ca\") pod \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") "
Jul 12 15:53:54.488350 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.488270    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-866sq\" (UniqueName: \"kubernetes.io/projected/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-kube-api-access-866sq\") pod \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") "
Jul 12 15:53:54.488350 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.488298    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls\") pod \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\" (UID: \"33c664cb-5a40-4ba1-ade5-5efe0dd8331d\") "
Jul 12 15:53:54.488578 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.488547    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33c664cb-5a40-4ba1-ade5-5efe0dd8331d" (UID: "33c664cb-5a40-4ba1-ade5-5efe0dd8331d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 15:53:54.490353 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.490324    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33c664cb-5a40-4ba1-ade5-5efe0dd8331d" (UID: "33c664cb-5a40-4ba1-ade5-5efe0dd8331d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 15:53:54.490482 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.490417    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-kube-api-access-866sq" (OuterVolumeSpecName: "kube-api-access-866sq") pod "33c664cb-5a40-4ba1-ade5-5efe0dd8331d" (UID: "33c664cb-5a40-4ba1-ade5-5efe0dd8331d"). InnerVolumeSpecName "kube-api-access-866sq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 15:53:54.589447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.589394    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-866sq\" (UniqueName: \"kubernetes.io/projected/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-kube-api-access-866sq\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:54.589447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.589441    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-maas-api-tls\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:54.589447 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:54.589451    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c664cb-5a40-4ba1-ade5-5efe0dd8331d-openshift-service-ca\") on node \"ip-10-0-142-238.ec2.internal\" DevicePath \"\""
Jul 12 15:53:55.278626 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:55.278592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx" event={"ID":"33c664cb-5a40-4ba1-ade5-5efe0dd8331d","Type":"ContainerDied","Data":"0280977aa1d9412cfe12aa909005bebacf8d8e893c93b546742c96a274ff7aab"}
Jul 12 15:53:55.278626 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:55.278624    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"
Jul 12 15:53:55.279139 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:55.278654    2578 scope.go:117] "RemoveContainer" containerID="6af5ec23618fa5dd5c35d5b978009b8c041c4e071320344865df043d359a4f64"
Jul 12 15:53:55.298999 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:55.298976    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"]
Jul 12 15:53:55.305038 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:55.305015    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b08e9f-74dc88f748-g77rx"]
Jul 12 15:53:56.155092 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:53:56.155059    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c664cb-5a40-4ba1-ade5-5efe0dd8331d" path="/var/lib/kubelet/pods/33c664cb-5a40-4ba1-ade5-5efe0dd8331d/volumes"
Jul 12 15:54:01.002921 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:01.002886    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d465c99c9-p4lzv_6aed2e46-26b8-46af-b648-6c1151c386fe/authorino/0.log"
Jul 12 15:54:06.555782 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:06.555752    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc_9dc0240a-d3d7-493c-93fe-89a5cb0b1216/copy-manifests/0.log"
Jul 12 15:54:06.562043 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:06.562020    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc_9dc0240a-d3d7-493c-93fe-89a5cb0b1216/manager/0.log"
Jul 12 15:54:07.908032 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:07.908002    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d465c99c9-p4lzv_6aed2e46-26b8-46af-b648-6c1151c386fe/authorino/0.log"
Jul 12 15:54:08.255878 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:08.255764    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d4d44_e1780c8d-7a72-49a4-9d6c-03014b656975/kuadrant-console-plugin/0.log"
Jul 12 15:54:08.710649 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:08.710611    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h824m_7161441b-fa0e-42fd-af31-97ade7f92adf/manager/0.log"
Jul 12 15:54:09.266262 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:09.266231    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fd5b5f7f7-2dmkl_59270970-f6ab-4790-ba2d-a6a7ec2ec646/kube-auth-proxy/0.log"
Jul 12 15:54:09.603483 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:09.603452    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5xhh8_39b93fb0-0ab9-4374-811e-0fb7228a13fa/payload-pre-processing/0.log"
Jul 12 15:54:10.154203 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:10.154173    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6sls7_9810d92e-8e80-4370-8163-4a85748267cc/main/0.log"
Jul 12 15:54:10.263480 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:10.263453    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-68kn5_06515ed3-4011-47cb-a5f6-db3799b3c54a/main/0.log"
Jul 12 15:54:17.181460 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:17.181420    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8tpsh_471cb232-a846-442d-8666-f9cafab871f0/global-pull-secret-syncer/0.log"
Jul 12 15:54:17.348041 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:17.348015    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sw5r9_3ab4a2e4-6527-4d42-94c5-5c4126d3732c/konnectivity-agent/0.log"
Jul 12 15:54:17.422623 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:17.422593    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-238.ec2.internal_a2150346244117c78ff93c3def3065ea/haproxy/0.log"
Jul 12 15:54:23.009609 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:23.009575    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d465c99c9-p4lzv_6aed2e46-26b8-46af-b648-6c1151c386fe/authorino/0.log"
Jul 12 15:54:23.094722 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:23.094688    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d4d44_e1780c8d-7a72-49a4-9d6c-03014b656975/kuadrant-console-plugin/0.log"
Jul 12 15:54:23.367496 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:23.367464    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h824m_7161441b-fa0e-42fd-af31-97ade7f92adf/manager/0.log"
Jul 12 15:54:25.046652 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.046621    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_573cd437-a92f-41dd-bf57-d5f61d7ea8b1/alertmanager/0.log"
Jul 12 15:54:25.075179 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.075153    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_573cd437-a92f-41dd-bf57-d5f61d7ea8b1/config-reloader/0.log"
Jul 12 15:54:25.099117 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.099096    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_573cd437-a92f-41dd-bf57-d5f61d7ea8b1/kube-rbac-proxy-web/0.log"
Jul 12 15:54:25.127353 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.127281    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_573cd437-a92f-41dd-bf57-d5f61d7ea8b1/kube-rbac-proxy/0.log"
Jul 12 15:54:25.154982 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.154961    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_573cd437-a92f-41dd-bf57-d5f61d7ea8b1/kube-rbac-proxy-metric/0.log"
Jul 12 15:54:25.189420 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.189397    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_573cd437-a92f-41dd-bf57-d5f61d7ea8b1/prom-label-proxy/0.log"
Jul 12 15:54:25.213184 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.213159    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_573cd437-a92f-41dd-bf57-d5f61d7ea8b1/init-config-reloader/0.log"
Jul 12 15:54:25.254420 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.254395    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-6rlp4_09036cb5-7b87-4026-bf52-146f44107558/cluster-monitoring-operator/0.log"
Jul 12 15:54:25.372071 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.372043    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-x7qh9_ac3bb11b-8b80-4c5f-98af-fc1700bab948/monitoring-plugin/0.log"
Jul 12 15:54:25.486690 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.486611    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8npnz_2801364e-bb09-487c-9f1b-2bbb4ae9c9a0/node-exporter/0.log"
Jul 12 15:54:25.502967 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.502944    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8npnz_2801364e-bb09-487c-9f1b-2bbb4ae9c9a0/kube-rbac-proxy/0.log"
Jul 12 15:54:25.523179 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.523159    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8npnz_2801364e-bb09-487c-9f1b-2bbb4ae9c9a0/init-textfile/0.log"
Jul 12 15:54:25.702404 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.702379    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1eee6dc8-f4d6-4e95-9ee2-635cdbb45528/prometheus/0.log"
Jul 12 15:54:25.730343 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.730321    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1eee6dc8-f4d6-4e95-9ee2-635cdbb45528/config-reloader/0.log"
Jul 12 15:54:25.750626 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.750567    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1eee6dc8-f4d6-4e95-9ee2-635cdbb45528/thanos-sidecar/0.log"
Jul 12 15:54:25.769966 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.769942    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1eee6dc8-f4d6-4e95-9ee2-635cdbb45528/kube-rbac-proxy-web/0.log"
Jul 12 15:54:25.790629 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.790607    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1eee6dc8-f4d6-4e95-9ee2-635cdbb45528/kube-rbac-proxy/0.log"
Jul 12 15:54:25.809484 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.809461    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1eee6dc8-f4d6-4e95-9ee2-635cdbb45528/kube-rbac-proxy-thanos/0.log"
Jul 12 15:54:25.834961 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.834940    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1eee6dc8-f4d6-4e95-9ee2-635cdbb45528/init-config-reloader/0.log"
Jul 12 15:54:25.911104 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.911081    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-qt6mv_62e21818-34d3-4c2e-beda-994c2fe01a47/prometheus-operator-admission-webhook/0.log"
Jul 12 15:54:25.940821 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.940778    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b6f595dcc-mqrs8_e705a0a9-f163-41de-8976-b9958cda7a89/telemeter-client/0.log"
Jul 12 15:54:25.960972 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.960949    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b6f595dcc-mqrs8_e705a0a9-f163-41de-8976-b9958cda7a89/reload/0.log"
Jul 12 15:54:25.981644 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:25.981624    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b6f595dcc-mqrs8_e705a0a9-f163-41de-8976-b9958cda7a89/kube-rbac-proxy/0.log"
Jul 12 15:54:26.009365 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:26.009302    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf7f4bb45-5hdf2_0d38f416-4fb6-4de7-8e67-b005f9be5f66/thanos-query/0.log"
Jul 12 15:54:26.032444 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:26.032420    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf7f4bb45-5hdf2_0d38f416-4fb6-4de7-8e67-b005f9be5f66/kube-rbac-proxy-web/0.log"
Jul 12 15:54:26.053839 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:26.053800    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf7f4bb45-5hdf2_0d38f416-4fb6-4de7-8e67-b005f9be5f66/kube-rbac-proxy/0.log"
Jul 12 15:54:26.079515 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:26.079225    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf7f4bb45-5hdf2_0d38f416-4fb6-4de7-8e67-b005f9be5f66/prom-label-proxy/0.log"
Jul 12 15:54:26.100004 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:26.099981    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf7f4bb45-5hdf2_0d38f416-4fb6-4de7-8e67-b005f9be5f66/kube-rbac-proxy-rules/0.log"
Jul 12 15:54:26.120851 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:26.120832    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf7f4bb45-5hdf2_0d38f416-4fb6-4de7-8e67-b005f9be5f66/kube-rbac-proxy-metrics/0.log"
Jul 12 15:54:27.343357 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:27.343329    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-mjwlp_b2f2c03a-fced-4df6-842b-257fca3c1f18/networking-console-plugin/0.log"
Jul 12 15:54:28.106393 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.106359    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"]
Jul 12 15:54:28.106937 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.106919    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c664cb-5a40-4ba1-ade5-5efe0dd8331d" containerName="maas-api"
Jul 12 15:54:28.106937 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.106938    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c664cb-5a40-4ba1-ade5-5efe0dd8331d" containerName="maas-api"
Jul 12 15:54:28.107068 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.106960    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a890e49b-1f52-4ff4-b6e0-42c60223831f" containerName="main"
Jul 12 15:54:28.107068 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.106967    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a890e49b-1f52-4ff4-b6e0-42c60223831f" containerName="main"
Jul 12 15:54:28.107068 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.106975    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="686e823e-6933-4a41-8e24-99fb9dbfc274" containerName="main"
Jul 12 15:54:28.107068 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.106982    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="686e823e-6933-4a41-8e24-99fb9dbfc274" containerName="main"
Jul 12 15:54:28.107068 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.106994    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ff1c33-674f-4a9b-bbe9-960ba247c049" containerName="maas-api"
Jul 12 15:54:28.107068 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.107000    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ff1c33-674f-4a9b-bbe9-960ba247c049" containerName="maas-api"
Jul 12 15:54:28.107068 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.107059    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="46ff1c33-674f-4a9b-bbe9-960ba247c049" containerName="maas-api"
Jul 12 15:54:28.107068 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.107068    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a890e49b-1f52-4ff4-b6e0-42c60223831f" containerName="main"
Jul 12 15:54:28.107335 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.107075    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="686e823e-6933-4a41-8e24-99fb9dbfc274" containerName="main"
Jul 12 15:54:28.107335 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.107084    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c664cb-5a40-4ba1-ade5-5efe0dd8331d" containerName="maas-api"
Jul 12 15:54:28.110411 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.110395    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.112576 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.112556    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n8bdw\"/\"default-dockercfg-fmjlr\""
Jul 12 15:54:28.112695 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.112679    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n8bdw\"/\"openshift-service-ca.crt\""
Jul 12 15:54:28.113717 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.113696    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n8bdw\"/\"kube-root-ca.crt\""
Jul 12 15:54:28.116389 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.116368    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"]
Jul 12 15:54:28.196624 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.196587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-lib-modules\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.196780 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.196633    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-sys\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.196780 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.196654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-proc\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.196780 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.196769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-podres\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.196930 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.196799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-658xl\" (UniqueName: \"kubernetes.io/projected/97c7af22-875d-460d-972a-221f26247d45-kube-api-access-658xl\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297399 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-proc\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297535 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-podres\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297535 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297456    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-658xl\" (UniqueName: \"kubernetes.io/projected/97c7af22-875d-460d-972a-221f26247d45-kube-api-access-658xl\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297535 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-lib-modules\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297535 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-proc\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297535 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297511    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-sys\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297740 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-sys\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297740 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297602    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-podres\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.297740 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.297630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97c7af22-875d-460d-972a-221f26247d45-lib-modules\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.304960 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.304938    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-658xl\" (UniqueName: \"kubernetes.io/projected/97c7af22-875d-460d-972a-221f26247d45-kube-api-access-658xl\") pod \"perf-node-gather-daemonset-dzmbj\" (UID: \"97c7af22-875d-460d-972a-221f26247d45\") " pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.421358 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.421275    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:28.441617 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.441586    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-655b7dc8c7-4hkcc_2c91dfc1-f39a-46d9-ba0b-345ba0a9863b/console/0.log"
Jul 12 15:54:28.550677 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:28.550655    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"]
Jul 12 15:54:28.552742 ip-10-0-142-238 kubenswrapper[2578]: W0712 15:54:28.552715    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod97c7af22_875d_460d_972a_221f26247d45.slice/crio-0c568edde711b52ccda47f535affec2b83efbb9c461878b89e9d3569315d873b WatchSource:0}: Error finding container 0c568edde711b52ccda47f535affec2b83efbb9c461878b89e9d3569315d873b: Status 404 returned error can't find the container with id 0c568edde711b52ccda47f535affec2b83efbb9c461878b89e9d3569315d873b
Jul 12 15:54:29.061601 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:29.061562    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-6l9p9_cbd2cf50-f068-4c6f-8f23-a254a4515790/volume-data-source-validator/0.log"
Jul 12 15:54:29.444016 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:29.443975    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj" event={"ID":"97c7af22-875d-460d-972a-221f26247d45","Type":"ContainerStarted","Data":"75f8ef4eac8cd4a11234b9a88a6e98f35812128b703edb241fb01e54b82b242c"}
Jul 12 15:54:29.444475 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:29.444066    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:29.444475 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:29.444085    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj" event={"ID":"97c7af22-875d-460d-972a-221f26247d45","Type":"ContainerStarted","Data":"0c568edde711b52ccda47f535affec2b83efbb9c461878b89e9d3569315d873b"}
Jul 12 15:54:29.459328 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:29.459279    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj" podStartSLOduration=1.459265886 podStartE2EDuration="1.459265886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:54:29.45818266 +0000 UTC m=+2687.825783507" watchObservedRunningTime="2026-07-12 15:54:29.459265886 +0000 UTC m=+2687.826866732"
Jul 12 15:54:30.161540 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:30.161510    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-smg4c_f904686e-b8e3-4353-af52-687df74b253a/dns/0.log"
Jul 12 15:54:30.182438 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:30.182415    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-smg4c_f904686e-b8e3-4353-af52-687df74b253a/kube-rbac-proxy/0.log"
Jul 12 15:54:30.252995 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:30.252963    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lqhsc_3e91ea4b-53f0-4d60-9398-f65b111d3fa0/dns-node-resolver/0.log"
Jul 12 15:54:30.854543 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:30.854515    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b4mrw_d1190c39-854f-4b29-b0de-b6b6e677834e/node-ca/0.log"
Jul 12 15:54:32.026836 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:32.026789    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fd5b5f7f7-2dmkl_59270970-f6ab-4790-ba2d-a6a7ec2ec646/kube-auth-proxy/0.log"
Jul 12 15:54:32.176020 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:32.175978    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5xhh8_39b93fb0-0ab9-4374-811e-0fb7228a13fa/payload-pre-processing/0.log"
Jul 12 15:54:32.981554 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:32.981527    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jp65g_c0a29124-e5db-412f-b3fe-5879f9d68c88/serve-healthcheck-canary/0.log"
Jul 12 15:54:33.481430 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:33.481400    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gk24p_2ceb397e-d9a7-470f-bd67-1c70333d4791/kube-rbac-proxy/0.log"
Jul 12 15:54:33.501974 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:33.501952    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gk24p_2ceb397e-d9a7-470f-bd67-1c70333d4791/exporter/0.log"
Jul 12 15:54:33.523279 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:33.523257    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gk24p_2ceb397e-d9a7-470f-bd67-1c70333d4791/extractor/0.log"
Jul 12 15:54:35.459255 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:35.459185    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n8bdw/perf-node-gather-daemonset-dzmbj"
Jul 12 15:54:36.621019 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:36.620990    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc_9dc0240a-d3d7-493c-93fe-89a5cb0b1216/manager/0.log"
Jul 12 15:54:36.640707 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:36.640683    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d5d6cd7fc-mlgnc_9dc0240a-d3d7-493c-93fe-89a5cb0b1216/copy-manifests/0.log"
Jul 12 15:54:38.202070 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:38.201968    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-676dd59b87-6nqkk_e501d677-e1e9-445a-a492-c0096e89a1d8/manager/0.log"
Jul 12 15:54:39.419289 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:39.419260    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fd5b5f7f7-2dmkl_59270970-f6ab-4790-ba2d-a6a7ec2ec646/kube-auth-proxy/0.log"
Jul 12 15:54:39.543695 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:39.543620    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5xhh8_39b93fb0-0ab9-4374-811e-0fb7228a13fa/payload-pre-processing/0.log"
Jul 12 15:54:41.027404 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:41.027369    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fd5b5f7f7-2dmkl_59270970-f6ab-4790-ba2d-a6a7ec2ec646/kube-auth-proxy/0.log"
Jul 12 15:54:41.155106 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:41.155077    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5xhh8_39b93fb0-0ab9-4374-811e-0fb7228a13fa/payload-pre-processing/0.log"
Jul 12 15:54:42.430686 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:42.430653    2578 scope.go:117] "RemoveContainer" containerID="5fb38974c47eb9b833be4c0780e849861e16fb6c99e0b4215ae22e033f2657f3"
Jul 12 15:54:43.350212 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:43.350188    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-bj8zt_c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d/migrator/0.log"
Jul 12 15:54:43.384545 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:43.384518    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-bj8zt_c33ab7fc-7329-4b24-8c51-7b2fc4f3fd0d/graceful-termination/0.log"
Jul 12 15:54:44.829140 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:44.829112    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gm7rc_22f293eb-9f03-46d2-8812-0ab424de95e3/kube-multus-additional-cni-plugins/0.log"
Jul 12 15:54:44.849775 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:44.849750    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gm7rc_22f293eb-9f03-46d2-8812-0ab424de95e3/egress-router-binary-copy/0.log"
Jul 12 15:54:44.869785 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:44.869755    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gm7rc_22f293eb-9f03-46d2-8812-0ab424de95e3/cni-plugins/0.log"
Jul 12 15:54:44.890058 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:44.890038    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gm7rc_22f293eb-9f03-46d2-8812-0ab424de95e3/bond-cni-plugin/0.log"
Jul 12 15:54:44.910590 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:44.910568    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gm7rc_22f293eb-9f03-46d2-8812-0ab424de95e3/routeoverride-cni/0.log"
Jul 12 15:54:44.931721 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:44.931695    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gm7rc_22f293eb-9f03-46d2-8812-0ab424de95e3/whereabouts-cni-bincopy/0.log"
Jul 12 15:54:44.953886 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:44.953857    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gm7rc_22f293eb-9f03-46d2-8812-0ab424de95e3/whereabouts-cni/0.log"
Jul 12 15:54:45.331461 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:45.331435    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-n9rp5_effea80b-8ac9-4862-86cf-e4f420b175a7/kube-multus/0.log"
Jul 12 15:54:45.436149 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:45.436115    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6vsjp_dcc058fa-537e-49c8-a4b9-79fe8986f083/network-metrics-daemon/0.log"
Jul 12 15:54:45.456268 ip-10-0-142-238 kubenswrapper[2578]: I0712 15:54:45.456246    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6vsjp_dcc058fa-537e-49c8-a4b9-79fe8986f083/kube-rbac-proxy/0.log"