Jul 15 10:17:57.925786 ip-10-0-130-114 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 10:17:57.925798 ip-10-0-130-114 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 10:17:57.925807 ip-10-0-130-114 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 10:17:57.926113 ip-10-0-130-114 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 10:18:08.058861 ip-10-0-130-114 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 10:18:08.058877 ip-10-0-130-114 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e6503a8726814fb49411d01bb1fb207c -- Jul 15 10:19:55.295589 ip-10-0-130-114 systemd[1]: Starting Kubernetes Kubelet... Jul 15 10:19:55.649484 ip-10-0-130-114 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 10:19:55.649484 ip-10-0-130-114 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 10:19:55.649484 ip-10-0-130-114 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 10:19:55.649484 ip-10-0-130-114 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 10:19:55.649967 ip-10-0-130-114 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 10:19:55.651763 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.651700 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 10:19:55.655165 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655151 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 10:19:55.655165 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655165 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655169 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655172 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655176 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655179 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655182 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655184 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655187 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655189 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655192 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655194 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655197 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655199 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655202 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655205 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655207 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655210 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655212 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655219 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655222 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 10:19:55.655225 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655224 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655227 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655230 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655233 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655235 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655238 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655241 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655243 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655246 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655248 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655251 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655253 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655256 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655258 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655261 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655264 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655266 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655268 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655271 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 10:19:55.655694 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655274 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655276 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655279 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655281 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655284 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655287 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655289 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655292 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655294 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655296 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655299 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655301 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655304 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655306 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655309 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655312 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655314 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655317 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655320 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 10:19:55.656136 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655322 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655325 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655339 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655342 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655346 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655350 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655353 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655357 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655361 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655364 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655366 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655369 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655372 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655375 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655378 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655383 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655386 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655389 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 10:19:55.656591 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655391 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655394 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655397 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655400 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655402 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655405 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655409 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655412 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655415 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655418 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655420 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655423 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655426 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655428 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655431 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655433 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655436 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655438 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655441 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655443 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655446 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 10:19:55.657043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.655448 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656099 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656105 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656108 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656111 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656113 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656116 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656120 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656122 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656125 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656127 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656130 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656133 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656135 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656137 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656140 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656143 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656145 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656149 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656151 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 10:19:55.657536 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656154 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656156 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656159 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656161 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656164 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656166 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656169 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656172 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656175 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656177 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656180 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656182 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656184 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656187 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656192 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656195 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656198 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656201 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656204 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656208 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 10:19:55.658008 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656210 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656213 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656215 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656218 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656221 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656224 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656227 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656229 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656232 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656234 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656237 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656239 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656242 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656244 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656246 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656249 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656251 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656254 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656257 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656259 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 10:19:55.658482 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656262 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656264 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656266 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656269 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656271 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656273 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656276 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656278 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656281 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656283 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656286 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656288 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656291 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656293 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656296 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656300 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656304 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656307 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656310 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 10:19:55.658944 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656313 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656316 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656318 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656321 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656323 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656340 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656342 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656345 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656348 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656350 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656353 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656355 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656358 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656360 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656362 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656371 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656375 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656377 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656380 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 10:19:55.659504 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656383 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656387 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656766 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656772 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656775 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656778 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656781 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656785 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656787 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656790 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656793 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656795 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656798 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656800 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656803 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656805 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656808 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656810 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656813 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656815 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 10:19:55.659953 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656817 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656820 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656822 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656825 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656827 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656830 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656832 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656835 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656837 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656840 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656842 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656844 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656848 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656852 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656855 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656857 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656860 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656863 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656866 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 10:19:55.660433 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656874 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656877 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656879 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656882 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656885 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656889 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656892 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656895 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656897 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656900 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656904 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656907 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656910 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656912 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656916 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656919 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656921 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656924 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656926 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 10:19:55.660886 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656929 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656931 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656934 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656936 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656939 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656943 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656946 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656948 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656951 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656953 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656955 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656958 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656960 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656962 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656970 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656973 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656976 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656978 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656981 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656983 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 10:19:55.661321 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656985 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656988 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656991 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656993 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656996 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.656998 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657000 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657003 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657005 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657008 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657011 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657013 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657016 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657018 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657020 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657022 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657025 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657029 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657032 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 10:19:55.661834 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657034 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657037 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657039 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657042 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657119 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657132 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657141 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657147 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657155 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657161 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657165 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657169 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657172 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657175 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657178 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657181 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657184 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657187 2585 flags.go:64] FLAG: --cgroup-root="" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657190 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657192 2585 flags.go:64] FLAG: --client-ca-file="" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657195 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657198 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657201 2585 flags.go:64] FLAG: --cluster-domain="" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657204 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 10:19:55.662276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657207 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657210 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657214 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657217 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657220 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657223 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657227 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657230 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657233 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657236 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657238 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657241 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657245 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657248 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657251 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657254 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657258 2585 flags.go:64] FLAG: --enable-server="true" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657262 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657266 2585 flags.go:64] FLAG: --event-burst="100" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657269 2585 flags.go:64] FLAG: --event-qps="50" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657272 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657275 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657278 2585 flags.go:64] FLAG: --eviction-hard="" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657281 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 10:19:55.662849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657284 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657287 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657289 2585 flags.go:64] FLAG: --eviction-soft="" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657292 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657295 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657298 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657300 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657303 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657306 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657308 2585 flags.go:64] FLAG: --feature-gates="" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657312 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657314 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657317 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657320 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657324 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657340 2585 flags.go:64] FLAG: --help="false" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657343 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-130-114.ec2.internal" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657346 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657349 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657352 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657356 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657359 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657362 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 10:19:55.663410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657364 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657368 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657371 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657374 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657377 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657381 2585 flags.go:64] FLAG: --kube-reserved="" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657384 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657386 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657389 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657392 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657395 2585 flags.go:64] FLAG: --lock-file="" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657398 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657400 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657404 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657409 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657412 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657414 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657417 2585 flags.go:64] FLAG: --logging-format="text" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657420 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657423 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657425 2585 flags.go:64] FLAG: --manifest-url="" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657429 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657433 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657437 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657441 2585 flags.go:64] FLAG: --max-pods="110" Jul 15 10:19:55.663965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657444 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657446 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657449 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657452 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657454 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657457 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657460 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657466 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657471 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657473 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657476 2585 flags.go:64] FLAG: --pod-cidr="" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657480 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657486 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657488 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657491 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657494 2585 flags.go:64] FLAG: --port="10250" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657497 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657499 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-074e1c308d84828fc" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657502 2585 flags.go:64] FLAG: --qos-reserved="" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657505 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657508 2585 flags.go:64] FLAG: --register-node="true" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657510 2585 flags.go:64] FLAG: --register-with-taints="" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657514 2585 flags.go:64] FLAG: --registry-burst="10" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657516 2585 flags.go:64] FLAG: --registry-qps="5" Jul 15 10:19:55.664555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657519 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657521 2585 flags.go:64] FLAG: --reserved-memory="" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657525 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657528 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657531 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657533 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657536 2585 flags.go:64] FLAG: --runonce="false" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657540 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657544 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657547 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657550 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657552 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657555 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657558 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657561 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657564 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657571 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657574 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657577 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657580 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657583 2585 flags.go:64] FLAG: --system-cgroups="" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657586 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657591 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657594 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657597 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 10:19:55.665144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657600 2585 flags.go:64] FLAG: --tls-min-version="" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657603 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657605 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657608 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657610 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657613 2585 flags.go:64] FLAG: --v="2" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657617 2585 flags.go:64] FLAG: --version="false" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657620 2585 flags.go:64] FLAG: --vmodule="" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657624 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.657626 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657741 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657746 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657749 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657752 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657756 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657758 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657761 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657763 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657765 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657768 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657771 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 10:19:55.665731 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657773 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657776 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657780 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657783 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657786 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657788 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657791 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657793 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657796 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657798 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657801 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657803 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657805 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657807 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657810 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657812 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657815 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657817 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657819 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657822 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 10:19:55.666224 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657825 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657827 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657830 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657832 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657835 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657837 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657840 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657842 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657844 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657847 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657849 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657851 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657854 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657856 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657860 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657863 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657865 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657867 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657870 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657872 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 10:19:55.666708 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657875 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657878 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657880 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657883 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657885 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657887 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657890 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657892 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657895 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657897 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657899 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657902 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657905 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657907 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657909 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657912 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657914 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657916 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657919 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657921 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 10:19:55.667178 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657923 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657926 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657929 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657931 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657933 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657936 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657939 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657942 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657944 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657946 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657948 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657951 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657953 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657957 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657960 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657963 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657965 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657968 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657971 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 10:19:55.667662 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657975 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657978 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657981 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657984 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657987 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657990 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657992 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657995 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.657997 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.658572 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.664759 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.664776 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.664794 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.664803 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664853 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 10:19:55.668113 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664857 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664861 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664863 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664866 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664869 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664871 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664874 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664877 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664879 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664882 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664885 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664887 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664890 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664893 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664895 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664898 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664900 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664903 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664905 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664908 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 10:19:55.668488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664910 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664913 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664915 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664918 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664920 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664923 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664925 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664928 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664931 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664934 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664936 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664939 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664941 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664943 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664946 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664949 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664951 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664954 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664957 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664959 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 10:19:55.668958 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664962 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664964 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664967 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664969 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664972 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664974 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664977 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664979 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664983 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664987 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664990 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664993 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664996 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.664998 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665001 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665004 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665006 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665009 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665011 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665014 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 10:19:55.669423 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665017 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665019 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665022 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665025 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665027 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665029 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665032 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665034 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665037 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665039 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665041 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665044 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665046 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665048 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665051 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665053 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665056 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665058 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665060 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665063 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 10:19:55.669888 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665065 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665068 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665070 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665074 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665078 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665080 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665083 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665085 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665087 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665089 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665092 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665094 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665097 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665101 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665104 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665108 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665111 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 10:19:55.670397 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665114 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.665118 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665221 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665225 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665228 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665230 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665233 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665235 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665238 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665240 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665243 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665245 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665248 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665250 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665253 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 10:19:55.670795 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665255 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665257 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665260 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665278 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665281 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665284 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665286 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665289 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665293 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665297 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665300 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665303 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665306 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665309 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665311 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665314 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665317 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665320 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665322 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 10:19:55.671316 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665324 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665342 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665345 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665347 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665350 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665352 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665354 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665357 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665360 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665362 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665364 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665367 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665369 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665372 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665374 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665376 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665379 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665382 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665384 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665386 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 10:19:55.672084 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665389 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665391 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665394 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665396 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665398 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665400 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665403 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665405 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665407 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665410 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665413 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665416 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665418 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665420 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665423 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665425 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665427 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665430 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665432 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665434 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 10:19:55.672711 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665437 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665439 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665441 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665444 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665446 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665448 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665450 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665453 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665455 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665458 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665460 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665462 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665465 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665467 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665470 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665472 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665475 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665477 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665480 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 10:19:55.673186 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665483 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665486 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665490 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665494 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665496 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665499 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665502 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:55.665504 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.665509 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.665594 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.667692 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.669836 2585 server.go:1019] "Starting client certificate rotation" Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.669932 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 10:19:55.673661 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.669972 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 10:19:55.673992 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.673812 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 10:19:55.675564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.675546 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 10:19:55.682689 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.682674 2585 log.go:25] "Validated CRI v1 runtime API" Jul 15 10:19:55.686764 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.686749 2585 log.go:25] "Validated CRI v1 image API" Jul 15 10:19:55.687469 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.687453 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 10:19:55.690827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.690810 2585 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8f5a2c58-82a5-48a4-a955-cfd4a31c017d:/dev/nvme0n1p4 f154ec72-cf2f-43ca-a33a-0f5f8e4bce62:/dev/nvme0n1p3] Jul 15 10:19:55.690928 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.690826 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 10:19:55.696309 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.696200 2585 manager.go:216] Machine: {Timestamp:2026-07-15 10:19:55.694990971 +0000 UTC m=+0.298716694 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3206561 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27dc800743433d4625f2f4f960e89d SystemUUID:ec27dc80-0743-433d-4625-f2f4f960e89d BootID:e6503a87-2681-4fb4-9411-d01bb1fb207c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d6:13:cf:cb:99 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d6:13:cf:cb:99 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:19:08:1d:73:09 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 10:19:55.696309 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.696303 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 10:19:55.696435 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.696388 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 10:19:55.699183 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.699111 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 10:19:55.699256 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.699174 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 10:19:55.699320 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.699188 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-114.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 10:19:55.699759 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.699749 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 10:19:55.699788 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.699761 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 10:19:55.699788 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.699768 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 10:19:55.700399 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.700389 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 10:19:55.701997 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.701986 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 10:19:55.705517 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.705505 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 15 10:19:55.705621 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.705612 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 10:19:55.705650 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.705640 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 15 10:19:55.705650 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.705648 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 10:19:55.705704 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.705657 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 10:19:55.705704 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.705666 2585 kubelet.go:398] "Adding apiserver pod source" Jul 15 10:19:55.705704 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.705676 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 10:19:55.705988 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.705974 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 10:19:55.706034 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.706000 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 10:19:55.708302 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.708285 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 10:19:55.710854 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.710837 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 10:19:55.710929 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.710868 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 10:19:55.711015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711005 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 10:19:55.711049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711019 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 10:19:55.711049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711025 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 10:19:55.711049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711030 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 10:19:55.711049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711037 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 10:19:55.711049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711041 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 10:19:55.711608 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711598 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 10:19:55.711608 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711607 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 10:19:55.711671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711614 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 10:19:55.711671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711619 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 10:19:55.711671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711626 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 10:19:55.711671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711633 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 10:19:55.711671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711653 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 10:19:55.711671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.711658 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 10:19:55.714233 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.714222 2585 server.go:1268] "Started kubelet" Jul 15 10:19:55.714377 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.714315 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 10:19:55.714450 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.714390 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 10:19:55.714586 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.714558 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 10:19:55.714937 ip-10-0-130-114 systemd[1]: Started Kubernetes Kubelet. Jul 15 10:19:55.716595 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.716563 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-114.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 10:19:55.716674 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.716595 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 10:19:55.716742 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.716691 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 10:19:55.717116 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.717101 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 15 10:19:55.718235 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.718215 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 10:19:55.719477 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.719460 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 10:19:55.722383 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.721491 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-114.ec2.internal.18c26f50ea7bf8dd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-114.ec2.internal,UID:ip-10-0-130-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-114.ec2.internal,},FirstTimestamp:2026-07-15 10:19:55.714197725 +0000 UTC m=+0.317923447,LastTimestamp:2026-07-15 10:19:55.714197725 +0000 UTC m=+0.317923447,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-114.ec2.internal,}" Jul 15 10:19:55.724612 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.724593 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 10:19:55.724612 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.724608 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 10:19:55.724779 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.724761 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:55.725270 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725241 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 10:19:55.725366 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725272 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 10:19:55.725366 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725308 2585 factory.go:55] Registering systemd factory Jul 15 10:19:55.725366 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725320 2585 factory.go:223] Registration of the systemd container factory successfully Jul 15 10:19:55.725366 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725238 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 10:19:55.725559 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725396 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 15 10:19:55.725559 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725403 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 15 10:19:55.725559 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725549 2585 factory.go:153] Registering CRI-O factory Jul 15 10:19:55.725684 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725562 2585 factory.go:223] Registration of the crio container factory successfully Jul 15 10:19:55.725684 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725632 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 10:19:55.725684 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725654 2585 factory.go:103] Registering Raw factory Jul 15 10:19:55.725684 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.725668 2585 manager.go:1196] Started watching for new ooms in manager Jul 15 10:19:55.726054 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.726036 2585 manager.go:349] Starting recovery of all containers Jul 15 10:19:55.727680 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.727640 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 10:19:55.727799 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.727782 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 10:19:55.737837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.737822 2585 manager.go:354] Recovery completed Jul 15 10:19:55.741676 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.741663 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 10:19:55.743183 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.743168 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 15 10:19:55.743266 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.743200 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 10:19:55.743266 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.743214 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 15 10:19:55.743804 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.743781 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 10:19:55.743804 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.743793 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 10:19:55.743804 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.743793 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 10:19:55.743948 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.743807 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 15 10:19:55.744458 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.744439 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 15 10:19:55.744526 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.744466 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 10:19:55.744526 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.744480 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 15 10:19:55.745088 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.745076 2585 policy_none.go:49] "None policy: Start" Jul 15 10:19:55.745120 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.745092 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 10:19:55.745120 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.745102 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 10:19:55.745365 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.745292 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-114.ec2.internal.18c26f50ec36425a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-114.ec2.internal,UID:ip-10-0-130-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-114.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-114.ec2.internal,},FirstTimestamp:2026-07-15 10:19:55.74318345 +0000 UTC m=+0.346909177,LastTimestamp:2026-07-15 10:19:55.74318345 +0000 UTC m=+0.346909177,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-114.ec2.internal,}" Jul 15 10:19:55.746127 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.746116 2585 policy_none.go:47] "Start" Jul 15 10:19:55.752050 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.752030 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hk5q9" Jul 15 10:19:55.754545 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.754490 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-114.ec2.internal.18c26f50ec369bb1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-114.ec2.internal,UID:ip-10-0-130-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-114.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-114.ec2.internal,},FirstTimestamp:2026-07-15 10:19:55.743206321 +0000 UTC m=+0.346932044,LastTimestamp:2026-07-15 10:19:55.743206321 +0000 UTC m=+0.346932044,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-114.ec2.internal,}" Jul 15 10:19:55.762072 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.762056 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hk5q9" Jul 15 10:19:55.765782 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.765728 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-114.ec2.internal.18c26f50ec36d107 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-114.ec2.internal,UID:ip-10-0-130-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-114.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-114.ec2.internal,},FirstTimestamp:2026-07-15 10:19:55.743219975 +0000 UTC m=+0.346945699,LastTimestamp:2026-07-15 10:19:55.743219975 +0000 UTC m=+0.346945699,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-114.ec2.internal,}" Jul 15 10:19:55.783810 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.783797 2585 manager.go:337] "Starting Device Plugin manager" Jul 15 10:19:55.783888 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.783823 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 10:19:55.783888 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.783832 2585 server.go:87] "Starting device plugin registration server" Jul 15 10:19:55.784029 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.784014 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 10:19:55.784072 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.784032 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 10:19:55.784135 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.784123 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 10:19:55.784234 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.784220 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 10:19:55.784234 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.784231 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 10:19:55.784665 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.784645 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 10:19:55.784736 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.784692 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:55.837979 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.837957 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 10:19:55.839179 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.839162 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 10:19:55.839241 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.839197 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 10:19:55.839241 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.839213 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 10:19:55.839359 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.839247 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 10:19:55.841864 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.841843 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 10:19:55.884895 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.884875 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 10:19:55.885630 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.885614 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 15 10:19:55.885699 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.885639 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 10:19:55.885699 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.885650 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 15 10:19:55.885699 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.885667 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-114.ec2.internal" Jul 15 10:19:55.895020 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.895004 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-114.ec2.internal" Jul 15 10:19:55.895065 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.895023 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-114.ec2.internal\": node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:55.924513 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.924470 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:55.940276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.940256 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal"] Jul 15 10:19:55.940341 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.940307 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 10:19:55.941077 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.941063 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 15 10:19:55.941147 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.941085 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 10:19:55.941147 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.941100 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 15 10:19:55.942198 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942186 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 10:19:55.942306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942292 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 15 10:19:55.942361 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942324 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 10:19:55.942822 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942807 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 15 10:19:55.942822 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942814 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 15 10:19:55.942911 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942837 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 10:19:55.942911 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942848 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 15 10:19:55.942911 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942837 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 10:19:55.942993 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.942913 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 15 10:19:55.944168 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.944154 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:55.944233 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.944179 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 10:19:55.944815 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.944799 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientMemory" Jul 15 10:19:55.944906 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.944829 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 10:19:55.944906 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:55.944842 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeHasSufficientPID" Jul 15 10:19:55.957862 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.957845 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-114.ec2.internal\" not found" node="ip-10-0-130-114.ec2.internal" Jul 15 10:19:55.961164 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:55.961147 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-114.ec2.internal\" not found" node="ip-10-0-130-114.ec2.internal" Jul 15 10:19:56.024678 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:56.024659 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:56.026873 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:56.026853 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:56.026956 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:56.026884 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4114c320ef9f8839a18178df34d18c8-config\") pod \"kube-apiserver-proxy-ip-10-0-130-114.ec2.internal\" (UID: \"b4114c320ef9f8839a18178df34d18c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 15 10:19:56.026956 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:56.026906 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.212994 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:57.212848 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:57.215055 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.215038 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4114c320ef9f8839a18178df34d18c8-config\") pod \"kube-apiserver-proxy-ip-10-0-130-114.ec2.internal\" (UID: \"b4114c320ef9f8839a18178df34d18c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.215121 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.215077 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.215121 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.215101 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.215194 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.215134 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4114c320ef9f8839a18178df34d18c8-config\") pod \"kube-apiserver-proxy-ip-10-0-130-114.ec2.internal\" (UID: \"b4114c320ef9f8839a18178df34d18c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.215194 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.215153 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.215194 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.215134 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3942009bd02b28965ce62ce1b271d1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal\" (UID: \"c3942009bd02b28965ce62ce1b271d1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.313518 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:57.313461 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:57.347793 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.347772 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.353016 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.352990 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.414504 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:57.414481 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:57.514988 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:57.514965 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:57.615480 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:57.615430 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:57.716102 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:57.716074 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:57.757372 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.757353 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 10:19:57.757711 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.757489 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 10:19:57.816179 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:57.816162 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-114.ec2.internal\" not found" Jul 15 10:19:57.853754 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.853725 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 10:14:56 +0000 UTC" deadline="2028-01-23 07:25:10.085748304 +0000 UTC" Jul 15 10:19:57.853754 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.853750 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13365h5m12.232001438s" Jul 15 10:19:57.888587 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.888568 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 10:19:57.890447 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:57.890425 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3942009bd02b28965ce62ce1b271d1a.slice/crio-8d51b085cf7195d0e84e1e544cbf8efaf9d377c0795705ced2b113b6e7063d4e WatchSource:0}: Error finding container 8d51b085cf7195d0e84e1e544cbf8efaf9d377c0795705ced2b113b6e7063d4e: Status 404 returned error can't find the container with id 8d51b085cf7195d0e84e1e544cbf8efaf9d377c0795705ced2b113b6e7063d4e Jul 15 10:19:57.890704 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:57.890693 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4114c320ef9f8839a18178df34d18c8.slice/crio-c7ecb7f1e1225659ecb013e73132ba6e4f1d399eb141d785c481bb23441249ac WatchSource:0}: Error finding container c7ecb7f1e1225659ecb013e73132ba6e4f1d399eb141d785c481bb23441249ac: Status 404 returned error can't find the container with id c7ecb7f1e1225659ecb013e73132ba6e4f1d399eb141d785c481bb23441249ac Jul 15 10:19:57.895022 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.895009 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 10:19:57.913097 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.913075 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.924413 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.924393 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 10:19:57.925758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.925744 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" Jul 15 10:19:57.930539 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.930505 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" event={"ID":"b4114c320ef9f8839a18178df34d18c8","Type":"ContainerStarted","Data":"c7ecb7f1e1225659ecb013e73132ba6e4f1d399eb141d785c481bb23441249ac"} Jul 15 10:19:57.931461 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.931442 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" event={"ID":"c3942009bd02b28965ce62ce1b271d1a","Type":"ContainerStarted","Data":"8d51b085cf7195d0e84e1e544cbf8efaf9d377c0795705ced2b113b6e7063d4e"} Jul 15 10:19:57.939586 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:57.939571 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 10:19:58.014350 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.014302 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 10:19:58.090863 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.090845 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 10:19:58.794273 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.794249 2585 apiserver.go:52] "Watching apiserver" Jul 15 10:19:58.801138 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.801114 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 10:19:58.802991 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.802954 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-zgjkv","openshift-network-operator/iptables-alerter-mjvnd","openshift-ovn-kubernetes/ovnkube-node-94td4","kube-system/global-pull-secret-syncer-6z7nq","kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal","openshift-dns/node-resolver-92kbz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal","openshift-multus/multus-p8dcx","openshift-multus/network-metrics-daemon-pdhc5","kube-system/konnectivity-agent-rl85z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29","openshift-cluster-node-tuning-operator/tuned-xkhts","openshift-image-registry/node-ca-j4p58","openshift-multus/multus-additional-cni-plugins-dmsg6"] Jul 15 10:19:58.804803 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.804784 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.805835 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.805813 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:19:58.805944 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.805888 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:19:58.807426 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.807405 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.807521 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.807466 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:19:58.807521 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.807479 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 10:19:58.807627 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.807520 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 10:19:58.808233 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.808003 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 10:19:58.808233 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.808053 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 10:19:58.808233 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.808126 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 10:19:58.808233 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.808212 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c94h4\"" Jul 15 10:19:58.808494 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.808472 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 10:19:58.808728 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.808713 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.810086 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.810053 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:19:58.810177 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.810127 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:19:58.810780 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.810678 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 10:19:58.810919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.810904 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 10:19:58.811377 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.811360 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.812208 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.812190 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-skvt5\"" Jul 15 10:19:58.812740 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.812715 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.813390 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.812984 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 10:19:58.813390 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.813223 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 10:19:58.813616 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.813598 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 10:19:58.813762 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.813747 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 10:19:58.813905 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.813888 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g4lzv\"" Jul 15 10:19:58.813973 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.813941 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 10:19:58.814429 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.814411 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:19:58.814913 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.814893 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 10:19:58.815014 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.814972 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gn9h8\"" Jul 15 10:19:58.815147 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.815012 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 10:19:58.815147 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.815040 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 10:19:58.815741 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.815722 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:58.816551 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.816529 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 10:19:58.816643 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.816562 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p44d5\"" Jul 15 10:19:58.816705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.816654 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 10:19:58.817410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.817394 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.818206 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.818188 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 10:19:58.818306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.818221 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 10:19:58.818306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.818188 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 10:19:58.818935 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.818917 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-txv7z\"" Jul 15 10:19:58.819117 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.819099 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:58.820271 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.820252 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rgg87\"" Jul 15 10:19:58.820418 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.820399 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 10:19:58.820607 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.820588 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 10:19:58.822024 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.822004 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 10:19:58.822294 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.822268 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 10:19:58.822433 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.822413 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hx8sx\"" Jul 15 10:19:58.822433 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.822429 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.822565 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.822457 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 10:19:58.823258 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823239 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-kubelet\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.823361 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823271 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-slash\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.823361 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823294 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-cni-bin\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.823471 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823346 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.823471 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823391 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d62dcd9-5681-4b2f-94d1-cecb85b68363-iptables-alerter-script\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.823471 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823413 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-ovn\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.823471 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823434 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-log-socket\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.823471 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823453 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/671b332f-6316-4d85-9309-ce6700e58de2-cni-binary-copy\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823474 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-cni-multus\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823494 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-hostroot\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823513 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d62dcd9-5681-4b2f-94d1-cecb85b68363-host-slash\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823533 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-cni-netd\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823552 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96f6262d-3033-449a-ade5-bc8c15fbf095-ovn-node-metrics-cert\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823572 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-ovnkube-script-lib\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823602 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d842f042-1ae5-4efe-a17a-7da9e8f76567-hosts-file\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823631 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-os-release\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823655 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-kubelet\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823675 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9188ae67-122d-4eae-aa25-98701dfe506b-agent-certs\") pod \"konnectivity-agent-rl85z\" (UID: \"9188ae67-122d-4eae-aa25-98701dfe506b\") " pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823715 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64dck\" (UniqueName: \"kubernetes.io/projected/aa664fba-4556-458c-9496-dc4cd0338f13-kube-api-access-64dck\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:19:58.823758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823745 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823786 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823811 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-ovnkube-config\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823835 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jphrr\" (UniqueName: \"kubernetes.io/projected/d842f042-1ae5-4efe-a17a-7da9e8f76567-kube-api-access-jphrr\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823858 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-cni-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823883 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-netns\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823903 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-etc-kubernetes\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823924 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c0b6c7bc-5908-4525-aacc-0b8c3186c646-kubelet-config\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823946 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-run-ovn-kubernetes\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.823988 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-socket-dir-parent\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824011 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-k8s-cni-cncf-io\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824047 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/671b332f-6316-4d85-9309-ce6700e58de2-multus-daemon-config\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824088 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-multus-certs\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824129 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rl4\" (UniqueName: \"kubernetes.io/projected/1d62dcd9-5681-4b2f-94d1-cecb85b68363-kube-api-access-g4rl4\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824154 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-systemd\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824194 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-var-lib-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824227 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-etc-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824256 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-env-overrides\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824276 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d842f042-1ae5-4efe-a17a-7da9e8f76567-tmp-dir\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824282 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824299 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-system-cni-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824327 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-cni-bin\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824365 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-systemd-units\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824399 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-run-netns\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824421 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-node-log\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824432 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824446 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4pmx\" (UniqueName: \"kubernetes.io/projected/96f6262d-3033-449a-ade5-bc8c15fbf095-kube-api-access-r4pmx\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824478 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lx6bh\"" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824481 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-cnibin\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824509 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-conf-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824532 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c8gs\" (UniqueName: \"kubernetes.io/projected/671b332f-6316-4d85-9309-ce6700e58de2-kube-api-access-4c8gs\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824556 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9188ae67-122d-4eae-aa25-98701dfe506b-konnectivity-ca\") pod \"konnectivity-agent-rl85z\" (UID: \"9188ae67-122d-4eae-aa25-98701dfe506b\") " pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824584 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824613 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c0b6c7bc-5908-4525-aacc-0b8c3186c646-dbus\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.824944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.824635 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:19:58.832019 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.832000 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 10:19:58.888322 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.888299 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k49nc" Jul 15 10:19:58.897898 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.897875 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k49nc" Jul 15 10:19:58.914699 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.914670 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 10:19:58.925394 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925339 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9188ae67-122d-4eae-aa25-98701dfe506b-konnectivity-ca\") pod \"konnectivity-agent-rl85z\" (UID: \"9188ae67-122d-4eae-aa25-98701dfe506b\") " pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:19:58.925489 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925400 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.925489 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-run-netns\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.925489 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925467 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-cnibin\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925489 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-host\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925510 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-serviceca\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925532 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925533 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-run-netns\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925557 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925579 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-sys-fs\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925601 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysconfig\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925611 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-cnibin\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.925639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925624 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvzwj\" (UniqueName: \"kubernetes.io/projected/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-kube-api-access-tvzwj\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925648 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cnibin\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925687 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-slash\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925717 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-registration-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925742 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8drsx\" (UniqueName: \"kubernetes.io/projected/d03907ff-8227-42dd-b009-8158ca057cad-kube-api-access-8drsx\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925764 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-tmp\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925793 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925839 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9188ae67-122d-4eae-aa25-98701dfe506b-konnectivity-ca\") pod \"konnectivity-agent-rl85z\" (UID: \"9188ae67-122d-4eae-aa25-98701dfe506b\") " pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925843 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-slash\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925866 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-log-socket\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925904 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-cni-multus\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925931 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d62dcd9-5681-4b2f-94d1-cecb85b68363-host-slash\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925903 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-log-socket\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925952 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-cni-multus\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925957 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925987 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-lib-modules\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.925994 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d62dcd9-5681-4b2f-94d1-cecb85b68363-host-slash\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.926082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926031 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96f6262d-3033-449a-ade5-bc8c15fbf095-ovn-node-metrics-cert\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926133 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-os-release\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926176 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-kubelet\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926214 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64dck\" (UniqueName: \"kubernetes.io/projected/aa664fba-4556-458c-9496-dc4cd0338f13-kube-api-access-64dck\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926266 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-kubelet\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926297 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c0b6c7bc-5908-4525-aacc-0b8c3186c646-kubelet-config\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926324 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-modprobe-d\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926347 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-run\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926381 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926411 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926438 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jphrr\" (UniqueName: \"kubernetes.io/projected/d842f042-1ae5-4efe-a17a-7da9e8f76567-kube-api-access-jphrr\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926461 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-cni-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926483 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-etc-kubernetes\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926537 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c0b6c7bc-5908-4525-aacc-0b8c3186c646-kubelet-config\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926542 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926574 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-etc-kubernetes\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926604 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-os-release\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.926782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926647 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926682 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926734 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-cni-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926793 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-systemd\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926845 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-os-release\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-run-ovn-kubernetes\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926940 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-multus-certs\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926972 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-device-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926986 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-run-ovn-kubernetes\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.926995 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-host\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927020 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-systemd\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927025 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-multus-certs\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927058 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-var-lib-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927084 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-systemd\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927107 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-etc-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927128 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-var-lib-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927163 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-env-overrides\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d842f042-1ae5-4efe-a17a-7da9e8f76567-tmp-dir\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927238 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-system-cni-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927262 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-cni-bin\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927290 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-systemd-units\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927373 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-var-lib-cni-bin\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927386 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-system-cni-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-node-log\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927438 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4pmx\" (UniqueName: \"kubernetes.io/projected/96f6262d-3033-449a-ade5-bc8c15fbf095-kube-api-access-r4pmx\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927463 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-conf-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c8gs\" (UniqueName: \"kubernetes.io/projected/671b332f-6316-4d85-9309-ce6700e58de2-kube-api-access-4c8gs\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927512 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c0b6c7bc-5908-4525-aacc-0b8c3186c646-dbus\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927577 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-socket-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927601 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d62dcd9-5681-4b2f-94d1-cecb85b68363-iptables-alerter-script\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927626 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-kubernetes\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927657 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-env-overrides\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.928397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysctl-conf\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927696 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-sys\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927726 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-kubelet\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927800 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-cni-bin\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927826 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-var-lib-kubelet\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927830 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d842f042-1ae5-4efe-a17a-7da9e8f76567-tmp-dir\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927860 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5sf2\" (UniqueName: \"kubernetes.io/projected/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-kube-api-access-c5sf2\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927869 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-kubelet\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927879 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-ovn\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927896 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/671b332f-6316-4d85-9309-ce6700e58de2-cni-binary-copy\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927910 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-hostroot\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927915 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c0b6c7bc-5908-4525-aacc-0b8c3186c646-dbus\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-cni-netd\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927940 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-ovnkube-script-lib\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927950 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-systemd-units\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927954 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d842f042-1ae5-4efe-a17a-7da9e8f76567-hosts-file\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927985 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d842f042-1ae5-4efe-a17a-7da9e8f76567-hosts-file\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.927983 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9188ae67-122d-4eae-aa25-98701dfe506b-agent-certs\") pod \"konnectivity-agent-rl85z\" (UID: \"9188ae67-122d-4eae-aa25-98701dfe506b\") " pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:19:58.929169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.928009 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysctl-d\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.928510 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-ovnkube-script-lib\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.928617 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.928661 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret podName:c0b6c7bc-5908-4525-aacc-0b8c3186c646 nodeName:}" failed. No retries permitted until 2026-07-15 10:19:59.42864789 +0000 UTC m=+2.944640366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret") pod "global-pull-secret-syncer-6z7nq" (UID: "c0b6c7bc-5908-4525-aacc-0b8c3186c646") : object "kube-system"/"original-pull-secret" not registered Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.928775 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-node-log\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.928911 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-conf-dir\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929154 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-etc-openvswitch\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.929228 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.929274 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs podName:aa664fba-4556-458c-9496-dc4cd0338f13 nodeName:}" failed. No retries permitted until 2026-07-15 10:19:59.429256693 +0000 UTC m=+2.945249176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs") pod "network-metrics-daemon-pdhc5" (UID: "aa664fba-4556-458c-9496-dc4cd0338f13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929310 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929310 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-cni-bin\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d62dcd9-5681-4b2f-94d1-cecb85b68363-iptables-alerter-script\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929343 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-ovnkube-config\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929369 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-netns\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929362 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-hostroot\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929406 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-netns\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929411 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-run-ovn\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929423 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96f6262d-3033-449a-ade5-bc8c15fbf095-host-cni-netd\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929486 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rl4\" (UniqueName: \"kubernetes.io/projected/1d62dcd9-5681-4b2f-94d1-cecb85b68363-kube-api-access-g4rl4\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929536 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-tuned\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929563 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-socket-dir-parent\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929607 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-k8s-cni-cncf-io\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929633 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/671b332f-6316-4d85-9309-ce6700e58de2-multus-daemon-config\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929658 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-system-cni-dir\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929686 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff89x\" (UniqueName: \"kubernetes.io/projected/7638a8fc-3b2c-4493-b111-a1e1fdac574c-kube-api-access-ff89x\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929811 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-multus-socket-dir-parent\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.929847 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96f6262d-3033-449a-ade5-bc8c15fbf095-ovnkube-config\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.930029 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/671b332f-6316-4d85-9309-ce6700e58de2-host-run-k8s-cni-cncf-io\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.930320 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96f6262d-3033-449a-ade5-bc8c15fbf095-ovn-node-metrics-cert\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.930437 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/671b332f-6316-4d85-9309-ce6700e58de2-multus-daemon-config\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.930766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.930477 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9188ae67-122d-4eae-aa25-98701dfe506b-agent-certs\") pod \"konnectivity-agent-rl85z\" (UID: \"9188ae67-122d-4eae-aa25-98701dfe506b\") " pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:19:58.932165 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.932147 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 10:19:58.932293 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.932170 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 10:19:58.932293 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.932183 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wnglt for pod openshift-network-diagnostics/network-check-target-zgjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:19:58.932293 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:58.932242 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt podName:ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d nodeName:}" failed. No retries permitted until 2026-07-15 10:19:59.432227174 +0000 UTC m=+2.948219641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wnglt" (UniqueName: "kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt") pod "network-check-target-zgjkv" (UID: "ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:19:58.932994 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.932953 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 10:19:58.934862 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.934844 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jphrr\" (UniqueName: \"kubernetes.io/projected/d842f042-1ae5-4efe-a17a-7da9e8f76567-kube-api-access-jphrr\") pod \"node-resolver-92kbz\" (UID: \"d842f042-1ae5-4efe-a17a-7da9e8f76567\") " pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:58.935531 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.935514 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64dck\" (UniqueName: \"kubernetes.io/projected/aa664fba-4556-458c-9496-dc4cd0338f13-kube-api-access-64dck\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:19:58.941168 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.941145 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/671b332f-6316-4d85-9309-ce6700e58de2-cni-binary-copy\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.943386 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.943361 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c8gs\" (UniqueName: \"kubernetes.io/projected/671b332f-6316-4d85-9309-ce6700e58de2-kube-api-access-4c8gs\") pod \"multus-p8dcx\" (UID: \"671b332f-6316-4d85-9309-ce6700e58de2\") " pod="openshift-multus/multus-p8dcx" Jul 15 10:19:58.943791 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.943766 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rl4\" (UniqueName: \"kubernetes.io/projected/1d62dcd9-5681-4b2f-94d1-cecb85b68363-kube-api-access-g4rl4\") pod \"iptables-alerter-mjvnd\" (UID: \"1d62dcd9-5681-4b2f-94d1-cecb85b68363\") " pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:58.944435 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:58.944414 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4pmx\" (UniqueName: \"kubernetes.io/projected/96f6262d-3033-449a-ade5-bc8c15fbf095-kube-api-access-r4pmx\") pod \"ovnkube-node-94td4\" (UID: \"96f6262d-3033-449a-ade5-bc8c15fbf095\") " pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:59.030433 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030408 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-serviceca\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:59.030519 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030444 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.030519 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030482 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-sys-fs\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.030519 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030501 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysconfig\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030522 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvzwj\" (UniqueName: \"kubernetes.io/projected/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-kube-api-access-tvzwj\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cnibin\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030565 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-registration-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030585 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8drsx\" (UniqueName: \"kubernetes.io/projected/d03907ff-8227-42dd-b009-8158ca057cad-kube-api-access-8drsx\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030604 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-tmp\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030615 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030627 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030660 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-sys-fs\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.030682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030670 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030689 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-lib-modules\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030715 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-registration-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030782 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030617 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysconfig\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030877 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-lib-modules\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030920 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cnibin\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030945 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-serviceca\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030966 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-modprobe-d\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.030990 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-run\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031017 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031039 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-systemd\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031074 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-os-release\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031090 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-modprobe-d\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031110 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-run\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031100 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-device-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031121 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-systemd\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031140 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-device-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031154 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-host\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031130 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031208 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-socket-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031210 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-os-release\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031222 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-host\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031253 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-kubernetes\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031279 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysctl-conf\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-sys\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031305 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d03907ff-8227-42dd-b009-8158ca057cad-socket-dir\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031323 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-kubernetes\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031326 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-var-lib-kubelet\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031353 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5sf2\" (UniqueName: \"kubernetes.io/projected/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-kube-api-access-c5sf2\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031360 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-sys\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031382 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-var-lib-kubelet\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031413 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysctl-conf\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031381 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysctl-d\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.031917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031444 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-sysctl-d\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031451 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031479 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-tuned\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031504 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-system-cni-dir\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031526 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff89x\" (UniqueName: \"kubernetes.io/projected/7638a8fc-3b2c-4493-b111-a1e1fdac574c-kube-api-access-ff89x\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031552 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031575 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-host\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031573 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7638a8fc-3b2c-4493-b111-a1e1fdac574c-system-cni-dir\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-host\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.031964 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.032714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.032045 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7638a8fc-3b2c-4493-b111-a1e1fdac574c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.033362 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.033341 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-tmp\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.033571 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.033535 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-etc-tuned\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.041113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.041088 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvzwj\" (UniqueName: \"kubernetes.io/projected/1dfb2620-41b7-49b6-aab2-dc73a0dca0a4-kube-api-access-tvzwj\") pod \"tuned-xkhts\" (UID: \"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.041629 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.041611 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff89x\" (UniqueName: \"kubernetes.io/projected/7638a8fc-3b2c-4493-b111-a1e1fdac574c-kube-api-access-ff89x\") pod \"multus-additional-cni-plugins-dmsg6\" (UID: \"7638a8fc-3b2c-4493-b111-a1e1fdac574c\") " pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.041629 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.041624 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8drsx\" (UniqueName: \"kubernetes.io/projected/d03907ff-8227-42dd-b009-8158ca057cad-kube-api-access-8drsx\") pod \"aws-ebs-csi-driver-node-7ss29\" (UID: \"d03907ff-8227-42dd-b009-8158ca057cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.041744 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.041626 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5sf2\" (UniqueName: \"kubernetes.io/projected/dd8b7c91-b49c-45bb-ae5b-e5ebe251de69-kube-api-access-c5sf2\") pod \"node-ca-j4p58\" (UID: \"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69\") " pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:59.117094 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.117031 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:19:59.124495 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.124476 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-92kbz" Jul 15 10:19:59.125593 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.125568 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f6262d_3033_449a_ade5_bc8c15fbf095.slice/crio-1f979c03da40b535781449d1f3f5274f595795b87756fe7c5d23b2344f8c7958 WatchSource:0}: Error finding container 1f979c03da40b535781449d1f3f5274f595795b87756fe7c5d23b2344f8c7958: Status 404 returned error can't find the container with id 1f979c03da40b535781449d1f3f5274f595795b87756fe7c5d23b2344f8c7958 Jul 15 10:19:59.132785 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.132765 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p8dcx" Jul 15 10:19:59.133079 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.133038 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd842f042_1ae5_4efe_a17a_7da9e8f76567.slice/crio-5c13541c79892b2fcf7cd5613e96326b26f7efd8845704f8ea2255d1b16ac469 WatchSource:0}: Error finding container 5c13541c79892b2fcf7cd5613e96326b26f7efd8845704f8ea2255d1b16ac469: Status 404 returned error can't find the container with id 5c13541c79892b2fcf7cd5613e96326b26f7efd8845704f8ea2255d1b16ac469 Jul 15 10:19:59.139856 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.139828 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mjvnd" Jul 15 10:19:59.140139 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.140115 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671b332f_6316_4d85_9309_ce6700e58de2.slice/crio-3faef4781008954b10fbda3b20d2e761be6fe85d3f00e291d8d9947063684d75 WatchSource:0}: Error finding container 3faef4781008954b10fbda3b20d2e761be6fe85d3f00e291d8d9947063684d75: Status 404 returned error can't find the container with id 3faef4781008954b10fbda3b20d2e761be6fe85d3f00e291d8d9947063684d75 Jul 15 10:19:59.144838 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.144819 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:19:59.147928 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.147905 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d62dcd9_5681_4b2f_94d1_cecb85b68363.slice/crio-485946fff71b17b5e7bfe7c1d2a7ad7fbaac81c21b8680a0dcfae2fc357cf0d3 WatchSource:0}: Error finding container 485946fff71b17b5e7bfe7c1d2a7ad7fbaac81c21b8680a0dcfae2fc357cf0d3: Status 404 returned error can't find the container with id 485946fff71b17b5e7bfe7c1d2a7ad7fbaac81c21b8680a0dcfae2fc357cf0d3 Jul 15 10:19:59.150671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.150489 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" Jul 15 10:19:59.153036 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.153012 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9188ae67_122d_4eae_aa25_98701dfe506b.slice/crio-5593cac0c515328de2621450dc7733f787b5213331cb65d721958c6d2869b714 WatchSource:0}: Error finding container 5593cac0c515328de2621450dc7733f787b5213331cb65d721958c6d2869b714: Status 404 returned error can't find the container with id 5593cac0c515328de2621450dc7733f787b5213331cb65d721958c6d2869b714 Jul 15 10:19:59.156994 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.156952 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xkhts" Jul 15 10:19:59.158982 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.158946 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd03907ff_8227_42dd_b009_8158ca057cad.slice/crio-01bf20267c96d2209bcf76ced17569deb46999ecb9e17d1214de6c81c9c49916 WatchSource:0}: Error finding container 01bf20267c96d2209bcf76ced17569deb46999ecb9e17d1214de6c81c9c49916: Status 404 returned error can't find the container with id 01bf20267c96d2209bcf76ced17569deb46999ecb9e17d1214de6c81c9c49916 Jul 15 10:19:59.162326 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.162308 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j4p58" Jul 15 10:19:59.168306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.168151 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" Jul 15 10:19:59.168883 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.168853 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dfb2620_41b7_49b6_aab2_dc73a0dca0a4.slice/crio-1361a3880ecec23fde1702802bd8ce422097417fe4cdc2ac44f4ea977709cc2f WatchSource:0}: Error finding container 1361a3880ecec23fde1702802bd8ce422097417fe4cdc2ac44f4ea977709cc2f: Status 404 returned error can't find the container with id 1361a3880ecec23fde1702802bd8ce422097417fe4cdc2ac44f4ea977709cc2f Jul 15 10:19:59.172606 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.172582 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8b7c91_b49c_45bb_ae5b_e5ebe251de69.slice/crio-63ced1c0a4e844b80d13676ff9484db2d949ce481fbf5e8d8f67b199e883f5ca WatchSource:0}: Error finding container 63ced1c0a4e844b80d13676ff9484db2d949ce481fbf5e8d8f67b199e883f5ca: Status 404 returned error can't find the container with id 63ced1c0a4e844b80d13676ff9484db2d949ce481fbf5e8d8f67b199e883f5ca Jul 15 10:19:59.177439 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:19:59.177415 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7638a8fc_3b2c_4493_b111_a1e1fdac574c.slice/crio-8fb9ee9c1c2aeb49dae6f7f3ed41219e91fd9774750b4a1e4f2534c1f6584011 WatchSource:0}: Error finding container 8fb9ee9c1c2aeb49dae6f7f3ed41219e91fd9774750b4a1e4f2534c1f6584011: Status 404 returned error can't find the container with id 8fb9ee9c1c2aeb49dae6f7f3ed41219e91fd9774750b4a1e4f2534c1f6584011 Jul 15 10:19:59.434372 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.434302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:19:59.434372 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.434351 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:19:59.434528 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.434399 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:19:59.434528 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.434430 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 10:19:59.434528 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.434494 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret podName:c0b6c7bc-5908-4525-aacc-0b8c3186c646 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:00.434475708 +0000 UTC m=+3.950468174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret") pod "global-pull-secret-syncer-6z7nq" (UID: "c0b6c7bc-5908-4525-aacc-0b8c3186c646") : object "kube-system"/"original-pull-secret" not registered Jul 15 10:19:59.434528 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.434526 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 10:19:59.434770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.434545 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 10:19:59.434770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.434556 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wnglt for pod openshift-network-diagnostics/network-check-target-zgjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:19:59.434770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.434599 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt podName:ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d nodeName:}" failed. No retries permitted until 2026-07-15 10:20:00.434585109 +0000 UTC m=+3.950577586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wnglt" (UniqueName: "kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt") pod "network-check-target-zgjkv" (UID: "ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:19:59.434770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.434661 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:19:59.434770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.434691 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs podName:aa664fba-4556-458c-9496-dc4cd0338f13 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:00.434683798 +0000 UTC m=+3.950676261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs") pod "network-metrics-daemon-pdhc5" (UID: "aa664fba-4556-458c-9496-dc4cd0338f13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:19:59.900733 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.900649 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 10:14:58 +0000 UTC" deadline="2028-03-08 10:20:33.78795506 +0000 UTC" Jul 15 10:19:59.900733 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.900694 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14448h0m33.887270975s" Jul 15 10:19:59.927350 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.927327 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:19:59.927483 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:19:59.927445 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:19:59.949543 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.949511 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"1f979c03da40b535781449d1f3f5274f595795b87756fe7c5d23b2344f8c7958"} Jul 15 10:19:59.959834 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.959803 2585 generic.go:356] "Generic (PLEG): container finished" podID="c3942009bd02b28965ce62ce1b271d1a" containerID="32b088a8e96b5347c19af07b95397803abb7610924c67128862a5f3de8561131" exitCode=0 Jul 15 10:19:59.959939 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.959889 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" event={"ID":"c3942009bd02b28965ce62ce1b271d1a","Type":"ContainerDied","Data":"32b088a8e96b5347c19af07b95397803abb7610924c67128862a5f3de8561131"} Jul 15 10:19:59.973044 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.972688 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" event={"ID":"7638a8fc-3b2c-4493-b111-a1e1fdac574c","Type":"ContainerStarted","Data":"8fb9ee9c1c2aeb49dae6f7f3ed41219e91fd9774750b4a1e4f2534c1f6584011"} Jul 15 10:19:59.976185 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.976121 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j4p58" event={"ID":"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69","Type":"ContainerStarted","Data":"63ced1c0a4e844b80d13676ff9484db2d949ce481fbf5e8d8f67b199e883f5ca"} Jul 15 10:19:59.980252 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.980190 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xkhts" event={"ID":"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4","Type":"ContainerStarted","Data":"1361a3880ecec23fde1702802bd8ce422097417fe4cdc2ac44f4ea977709cc2f"} Jul 15 10:19:59.990785 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.989957 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mjvnd" event={"ID":"1d62dcd9-5681-4b2f-94d1-cecb85b68363","Type":"ContainerStarted","Data":"485946fff71b17b5e7bfe7c1d2a7ad7fbaac81c21b8680a0dcfae2fc357cf0d3"} Jul 15 10:19:59.999212 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:19:59.999143 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-92kbz" event={"ID":"d842f042-1ae5-4efe-a17a-7da9e8f76567","Type":"ContainerStarted","Data":"5c13541c79892b2fcf7cd5613e96326b26f7efd8845704f8ea2255d1b16ac469"} Jul 15 10:20:00.011579 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.011551 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" event={"ID":"d03907ff-8227-42dd-b009-8158ca057cad","Type":"ContainerStarted","Data":"01bf20267c96d2209bcf76ced17569deb46999ecb9e17d1214de6c81c9c49916"} Jul 15 10:20:00.020388 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.020362 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rl85z" event={"ID":"9188ae67-122d-4eae-aa25-98701dfe506b","Type":"ContainerStarted","Data":"5593cac0c515328de2621450dc7733f787b5213331cb65d721958c6d2869b714"} Jul 15 10:20:00.025361 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.025332 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p8dcx" event={"ID":"671b332f-6316-4d85-9309-ce6700e58de2","Type":"ContainerStarted","Data":"3faef4781008954b10fbda3b20d2e761be6fe85d3f00e291d8d9947063684d75"} Jul 15 10:20:00.441171 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.441137 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:00.441332 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.441202 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:00.441332 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.441236 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:00.441458 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.441360 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:00.441458 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.441417 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs podName:aa664fba-4556-458c-9496-dc4cd0338f13 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:02.441398867 +0000 UTC m=+5.957391334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs") pod "network-metrics-daemon-pdhc5" (UID: "aa664fba-4556-458c-9496-dc4cd0338f13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:00.441876 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.441856 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 10:20:00.441943 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.441881 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 10:20:00.441943 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.441895 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wnglt for pod openshift-network-diagnostics/network-check-target-zgjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:00.441943 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.441939 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt podName:ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d nodeName:}" failed. No retries permitted until 2026-07-15 10:20:02.441925573 +0000 UTC m=+5.957918039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wnglt" (UniqueName: "kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt") pod "network-check-target-zgjkv" (UID: "ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:00.442114 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.442004 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:00.442114 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.442039 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret podName:c0b6c7bc-5908-4525-aacc-0b8c3186c646 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:02.442027971 +0000 UTC m=+5.958020454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret") pod "global-pull-secret-syncer-6z7nq" (UID: "c0b6c7bc-5908-4525-aacc-0b8c3186c646") : object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:00.901243 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.901137 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 10:14:58 +0000 UTC" deadline="2028-04-27 12:48:43.731017768 +0000 UTC" Jul 15 10:20:00.901243 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.901193 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15650h28m42.829827642s" Jul 15 10:20:00.930308 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.930278 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:00.930491 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.930400 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:00.930805 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:00.930764 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:00.930907 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:00.930849 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:01.927669 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:01.927638 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:01.928131 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:01.927758 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:02.459489 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:02.459456 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:02.459657 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.459611 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:02.459720 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.459680 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret podName:c0b6c7bc-5908-4525-aacc-0b8c3186c646 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:06.459660871 +0000 UTC m=+9.975653341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret") pod "global-pull-secret-syncer-6z7nq" (UID: "c0b6c7bc-5908-4525-aacc-0b8c3186c646") : object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:02.459886 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:02.459857 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:02.459963 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:02.459928 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:02.460188 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.460036 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:02.460188 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.460038 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 10:20:02.460188 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.460110 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs podName:aa664fba-4556-458c-9496-dc4cd0338f13 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:06.460086873 +0000 UTC m=+9.976079362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs") pod "network-metrics-daemon-pdhc5" (UID: "aa664fba-4556-458c-9496-dc4cd0338f13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:02.460188 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.460128 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 10:20:02.460188 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.460143 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wnglt for pod openshift-network-diagnostics/network-check-target-zgjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:02.460188 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.460179 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt podName:ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d nodeName:}" failed. No retries permitted until 2026-07-15 10:20:06.460168686 +0000 UTC m=+9.976161171 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wnglt" (UniqueName: "kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt") pod "network-check-target-zgjkv" (UID: "ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:02.928691 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:02.928611 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:02.929114 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.928755 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:02.929114 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:02.928808 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:02.929114 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:02.928922 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:03.927335 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:03.927302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:03.927529 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:03.927437 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:04.928084 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:04.928029 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:04.928084 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:04.928051 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:04.928639 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:04.928179 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:04.928639 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:04.928272 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:05.927473 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:05.927419 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:05.927652 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:05.927546 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:06.492393 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:06.492351 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:06.492422 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:06.492452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.492542 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.492604 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs podName:aa664fba-4556-458c-9496-dc4cd0338f13 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:14.492584958 +0000 UTC m=+18.008577438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs") pod "network-metrics-daemon-pdhc5" (UID: "aa664fba-4556-458c-9496-dc4cd0338f13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.492600 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.492666 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret podName:c0b6c7bc-5908-4525-aacc-0b8c3186c646 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:14.492648816 +0000 UTC m=+18.008641493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret") pod "global-pull-secret-syncer-6z7nq" (UID: "c0b6c7bc-5908-4525-aacc-0b8c3186c646") : object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.492544 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.492701 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.492715 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wnglt for pod openshift-network-diagnostics/network-check-target-zgjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:06.492835 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.492749 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt podName:ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d nodeName:}" failed. No retries permitted until 2026-07-15 10:20:14.492739247 +0000 UTC m=+18.008731714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wnglt" (UniqueName: "kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt") pod "network-check-target-zgjkv" (UID: "ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:06.929269 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:06.928601 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:06.929269 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.928710 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:06.929269 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:06.929079 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:06.929269 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:06.929174 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:07.927592 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:07.927560 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:07.928036 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:07.927684 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:08.927254 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:08.927221 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:08.927462 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:08.927349 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:08.927462 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:08.927415 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:08.927564 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:08.927492 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:09.927574 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:09.927547 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:09.928005 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:09.927652 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:10.928080 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:10.928037 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:10.928528 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:10.928198 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:10.928528 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:10.928256 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:10.928528 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:10.928349 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:11.927480 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:11.927450 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:11.927660 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:11.927566 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:12.927927 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:12.927857 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:12.928339 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:12.927869 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:12.928339 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:12.927977 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:12.928339 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:12.928057 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:13.927668 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:13.927630 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:13.927891 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:13.927762 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:14.560408 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:14.560366 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.560510 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.560538 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.560551 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wnglt for pod openshift-network-diagnostics/network-check-target-zgjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.560586 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.560604 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt podName:ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d nodeName:}" failed. No retries permitted until 2026-07-15 10:20:30.560587244 +0000 UTC m=+34.076579723 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wnglt" (UniqueName: "kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt") pod "network-check-target-zgjkv" (UID: "ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.560621 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret podName:c0b6c7bc-5908-4525-aacc-0b8c3186c646 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:30.560610974 +0000 UTC m=+34.076603451 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret") pod "global-pull-secret-syncer-6z7nq" (UID: "c0b6c7bc-5908-4525-aacc-0b8c3186c646") : object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:14.560511 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:14.560658 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.560747 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:14.560851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.560786 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs podName:aa664fba-4556-458c-9496-dc4cd0338f13 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:30.560774367 +0000 UTC m=+34.076766843 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs") pod "network-metrics-daemon-pdhc5" (UID: "aa664fba-4556-458c-9496-dc4cd0338f13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:14.927710 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:14.927634 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:14.927869 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.927769 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:14.927869 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:14.927823 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:14.927982 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:14.927930 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:15.927913 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:15.927880 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:15.928322 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:15.927983 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:16.928996 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:16.928967 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:16.929355 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:16.929096 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:16.929355 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:16.929174 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:16.929355 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:16.929280 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:17.927127 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:17.927103 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:17.927228 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:17.927183 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:18.072591 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:18.072527 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" event={"ID":"b4114c320ef9f8839a18178df34d18c8","Type":"ContainerStarted","Data":"757ccfee8c942a31113d710881bd965a319c275c153f9e2eaf1320a2cac75ef9"} Jul 15 10:20:18.095146 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:18.094741 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" event={"ID":"c3942009bd02b28965ce62ce1b271d1a","Type":"ContainerStarted","Data":"dfabdc00b47225f3a188704cdd83885bbbb72baea0b231a052b0b0f5b7eecd9c"} Jul 15 10:20:18.117892 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:18.116880 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-114.ec2.internal" podStartSLOduration=21.11686051 podStartE2EDuration="21.11686051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:18.095614144 +0000 UTC m=+21.611606628" watchObservedRunningTime="2026-07-15 10:20:18.11686051 +0000 UTC m=+21.632852995" Jul 15 10:20:18.117892 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:18.117011 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-114.ec2.internal" podStartSLOduration=21.117004512 podStartE2EDuration="21.117004512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:18.116409543 +0000 UTC m=+21.632402029" watchObservedRunningTime="2026-07-15 10:20:18.117004512 +0000 UTC m=+21.632996997" Jul 15 10:20:18.927698 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:18.927488 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:18.927827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:18.927537 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:18.927827 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:18.927757 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:18.927827 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:18.927810 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:19.106697 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.106671 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"0fce35acee506ef7e9b2b2cf2d7272ee669be13b86fe3972220494070424c7e7"} Jul 15 10:20:19.107388 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.106734 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"d23f5ca7eab5876334e6a40a5cf730481333d6f1c2bee26a67cb740a969e3ebc"} Jul 15 10:20:19.107388 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.106782 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"4101af0025ccf7be335130b6610e45c074374909608a9eb9af9251a1c6986fe4"} Jul 15 10:20:19.107388 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.106811 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"de718ccebc3b4d173babf03e8319a77ac9c778a6704f31c454aff5fb509cf078"} Jul 15 10:20:19.107388 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.106839 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"85d47ab1d344b9520e4f52f0e5102f243a34c992a9131a111eb03b19160392af"} Jul 15 10:20:19.107388 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.106883 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"3bccbf44f97eabc0097cd4f3344f0a835fc56fc4e8c644b1f4eed3efffb2802c"} Jul 15 10:20:19.108121 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.108099 2585 generic.go:356] "Generic (PLEG): container finished" podID="7638a8fc-3b2c-4493-b111-a1e1fdac574c" containerID="24775ad5b10f129d17feddb49d20f6b54ae9b68acb578863a384dececc6d1cca" exitCode=0 Jul 15 10:20:19.108215 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.108168 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" event={"ID":"7638a8fc-3b2c-4493-b111-a1e1fdac574c","Type":"ContainerDied","Data":"24775ad5b10f129d17feddb49d20f6b54ae9b68acb578863a384dececc6d1cca"} Jul 15 10:20:19.109514 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.109486 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j4p58" event={"ID":"dd8b7c91-b49c-45bb-ae5b-e5ebe251de69","Type":"ContainerStarted","Data":"a059db9b8583a300a4db7ea9be46f18a144316090732b8e6adc5b5cbcb5d442e"} Jul 15 10:20:19.110908 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.110874 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xkhts" event={"ID":"1dfb2620-41b7-49b6-aab2-dc73a0dca0a4","Type":"ContainerStarted","Data":"1ce9e75d40fba6c6f4ff3f906e927f0b27d6b7b88a3ddcf4e01c0d07550bff78"} Jul 15 10:20:19.112170 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.112147 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-92kbz" event={"ID":"d842f042-1ae5-4efe-a17a-7da9e8f76567","Type":"ContainerStarted","Data":"3d7d1f7d23d2e17d412bdcba28ef8679d3165a989141dbac412d1856122ada98"} Jul 15 10:20:19.113525 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.113496 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" event={"ID":"d03907ff-8227-42dd-b009-8158ca057cad","Type":"ContainerStarted","Data":"b89a303b50c7a366292979d89f52ce24b6fc7970ca240fe0d21121b35f321627"} Jul 15 10:20:19.115029 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.115009 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rl85z" event={"ID":"9188ae67-122d-4eae-aa25-98701dfe506b","Type":"ContainerStarted","Data":"56e72d3f4f6d6b2484108e0e7a2e1a71930fdfcf626e0735c7969c30ec50bac6"} Jul 15 10:20:19.116352 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.116333 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p8dcx" event={"ID":"671b332f-6316-4d85-9309-ce6700e58de2","Type":"ContainerStarted","Data":"08132572c0ead442d1dfe48f08592afcef191a8ca563a35691014259a58b1d98"} Jul 15 10:20:19.135016 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.134975 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j4p58" podStartSLOduration=8.658858164 podStartE2EDuration="22.134964787s" totalImagesPullingTime="13.476106623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:18.136617728 +0000 UTC m=+21.652610214" watchObservedRunningTime="2026-07-15 10:20:19.134964787 +0000 UTC m=+22.650957302" Jul 15 10:20:19.153806 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.153785 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:20:19.154403 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.154389 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:20:19.156812 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.156783 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p8dcx" podStartSLOduration=3.315173804 podStartE2EDuration="22.156772966s" totalImagesPullingTime="18.841599162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:19.156687778 +0000 UTC m=+22.672680264" watchObservedRunningTime="2026-07-15 10:20:19.156772966 +0000 UTC m=+22.672765451" Jul 15 10:20:19.178201 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.178149 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rl85z" podStartSLOduration=3.458753667 podStartE2EDuration="22.178134629s" totalImagesPullingTime="18.719380962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:19.177864254 +0000 UTC m=+22.693856738" watchObservedRunningTime="2026-07-15 10:20:19.178134629 +0000 UTC m=+22.694127114" Jul 15 10:20:19.196011 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.195976 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-92kbz" podStartSLOduration=3.419380323 podStartE2EDuration="22.195963088s" totalImagesPullingTime="18.776582765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:19.19589417 +0000 UTC m=+22.711886655" watchObservedRunningTime="2026-07-15 10:20:19.195963088 +0000 UTC m=+22.711955575" Jul 15 10:20:19.238258 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.238226 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xkhts" podStartSLOduration=3.495764219 podStartE2EDuration="22.238212128s" totalImagesPullingTime="18.742447909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:19.217052581 +0000 UTC m=+22.733045077" watchObservedRunningTime="2026-07-15 10:20:19.238212128 +0000 UTC m=+22.754204614" Jul 15 10:20:19.436322 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.436282 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 10:20:19.883672 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.883472 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T10:20:19.436295622Z","UUID":"29db04cd-44f2-43ee-807f-35ddc13ec754","Handler":null,"Name":"","Endpoint":""} Jul 15 10:20:19.885214 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.885189 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 10:20:19.885315 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.885237 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 10:20:19.927524 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:19.927501 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:19.927623 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:19.927580 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:20.120389 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:20.120353 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mjvnd" event={"ID":"1d62dcd9-5681-4b2f-94d1-cecb85b68363","Type":"ContainerStarted","Data":"0c7abf9ff17a1b1c3dca164830da112799bfa43a2aa292ca9d5be87e04a42c21"} Jul 15 10:20:20.122522 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:20.122485 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" event={"ID":"d03907ff-8227-42dd-b009-8158ca057cad","Type":"ContainerStarted","Data":"bba278597fe04fbfe31cbe0e5acedd8f9134903898da6cbfda27fd14e805960e"} Jul 15 10:20:20.143377 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:20.143304 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mjvnd" podStartSLOduration=4.419213253 podStartE2EDuration="23.143292248s" totalImagesPullingTime="18.724078995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:20.143159094 +0000 UTC m=+23.659151599" watchObservedRunningTime="2026-07-15 10:20:20.143292248 +0000 UTC m=+23.659284733" Jul 15 10:20:20.927845 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:20.927776 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:20.928079 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:20.927906 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:20.928079 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:20.927958 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:20.928220 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:20.928091 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:21.128269 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:21.128078 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"26f3482475a343184a01bce4a55a091754136b0cb5c6e8f7d434bf29060e377a"} Jul 15 10:20:21.128602 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:21.128277 2585 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 10:20:21.927140 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:21.927086 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:21.927345 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:21.927218 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:22.132858 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:22.132828 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" event={"ID":"d03907ff-8227-42dd-b009-8158ca057cad","Type":"ContainerStarted","Data":"e2cee70d8ebf54f7ac89b27565fca8464d944e56be6758d078d198152d095a3f"} Jul 15 10:20:22.167895 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:22.167853 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ss29" podStartSLOduration=3.660742858 podStartE2EDuration="25.167841667s" totalImagesPullingTime="21.507098809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:22.166411039 +0000 UTC m=+25.682403523" watchObservedRunningTime="2026-07-15 10:20:22.167841667 +0000 UTC m=+25.683834152" Jul 15 10:20:22.576996 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:22.576956 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:20:22.577180 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:22.577125 2585 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 10:20:22.577546 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:22.577522 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rl85z" Jul 15 10:20:22.927863 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:22.927840 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:22.927987 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:22.927969 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:22.928056 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:22.928003 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:22.928146 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:22.928109 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:23.139438 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:23.139211 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" event={"ID":"96f6262d-3033-449a-ade5-bc8c15fbf095","Type":"ContainerStarted","Data":"3e0e9b93401429256c6a981127a6fd7f5fdbe3e5ae4f5e69fdbb7de09c7a96ea"} Jul 15 10:20:23.140268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:23.139709 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:20:23.140268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:23.139780 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:20:23.157155 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:23.157121 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:20:23.170716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:23.170669 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" podStartSLOduration=8.355415014 podStartE2EDuration="27.170655333s" totalImagesPullingTime="18.815240319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:23.170247747 +0000 UTC m=+26.686240232" watchObservedRunningTime="2026-07-15 10:20:23.170655333 +0000 UTC m=+26.686647817" Jul 15 10:20:23.927690 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:23.927661 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:23.927886 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:23.927748 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:24.142277 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:24.142244 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:20:24.157753 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:24.157728 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:20:24.927490 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:24.927464 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:24.927636 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:24.927471 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:24.927636 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:24.927595 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:24.927750 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:24.927649 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:25.145946 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:25.145913 2585 generic.go:356] "Generic (PLEG): container finished" podID="7638a8fc-3b2c-4493-b111-a1e1fdac574c" containerID="990718cfaedcf93a0c5055decbc7bf9979ba29a3c95384536fbe0cfc9ba1ab04" exitCode=0 Jul 15 10:20:25.146344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:25.145983 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" event={"ID":"7638a8fc-3b2c-4493-b111-a1e1fdac574c","Type":"ContainerDied","Data":"990718cfaedcf93a0c5055decbc7bf9979ba29a3c95384536fbe0cfc9ba1ab04"} Jul 15 10:20:25.406240 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:25.406177 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6z7nq"] Jul 15 10:20:25.406347 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:25.406270 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:25.406413 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:25.406370 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:25.409531 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:25.409506 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zgjkv"] Jul 15 10:20:25.409625 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:25.409593 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:25.409671 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:25.409657 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:25.411952 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:25.411933 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pdhc5"] Jul 15 10:20:25.412039 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:25.412001 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:25.412117 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:25.412100 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:26.929075 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:26.929029 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:26.929535 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:26.929133 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:26.929535 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:26.929163 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:26.929535 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:26.929193 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:26.929535 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:26.929212 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:26.929535 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:26.929260 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:27.155192 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:27.155163 2585 generic.go:356] "Generic (PLEG): container finished" podID="7638a8fc-3b2c-4493-b111-a1e1fdac574c" containerID="fc5b1f8e55a2fff908c822f3d7fa90b34c2bc0645a0dee8c1db23e7b4d8f3aef" exitCode=0 Jul 15 10:20:27.155312 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:27.155198 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" event={"ID":"7638a8fc-3b2c-4493-b111-a1e1fdac574c","Type":"ContainerDied","Data":"fc5b1f8e55a2fff908c822f3d7fa90b34c2bc0645a0dee8c1db23e7b4d8f3aef"} Jul 15 10:20:28.159656 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:28.159600 2585 generic.go:356] "Generic (PLEG): container finished" podID="7638a8fc-3b2c-4493-b111-a1e1fdac574c" containerID="975d2ec584a1c54d713d8972a4f3613158c0859d2b8ca84495b1a580b82d2ac7" exitCode=0 Jul 15 10:20:28.159656 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:28.159643 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" event={"ID":"7638a8fc-3b2c-4493-b111-a1e1fdac574c","Type":"ContainerDied","Data":"975d2ec584a1c54d713d8972a4f3613158c0859d2b8ca84495b1a580b82d2ac7"} Jul 15 10:20:28.928114 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:28.928087 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:28.928299 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:28.928080 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:28.928299 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:28.928200 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:28.928299 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:28.928273 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:28.928299 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:28.928080 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:28.928493 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:28.928371 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:30.586759 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:30.586725 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:30.586759 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:30.586764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.586850 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.586857 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:30.586888 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.586937 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs podName:aa664fba-4556-458c-9496-dc4cd0338f13 nodeName:}" failed. No retries permitted until 2026-07-15 10:21:02.586885815 +0000 UTC m=+66.102878296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs") pod "network-metrics-daemon-pdhc5" (UID: "aa664fba-4556-458c-9496-dc4cd0338f13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.586978 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.586993 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.587005 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wnglt for pod openshift-network-diagnostics/network-check-target-zgjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.587015 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret podName:c0b6c7bc-5908-4525-aacc-0b8c3186c646 nodeName:}" failed. No retries permitted until 2026-07-15 10:21:02.586995631 +0000 UTC m=+66.102988104 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret") pod "global-pull-secret-syncer-6z7nq" (UID: "c0b6c7bc-5908-4525-aacc-0b8c3186c646") : object "kube-system"/"original-pull-secret" not registered Jul 15 10:20:30.587264 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.587042 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt podName:ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d nodeName:}" failed. No retries permitted until 2026-07-15 10:21:02.587032241 +0000 UTC m=+66.103024722 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wnglt" (UniqueName: "kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt") pod "network-check-target-zgjkv" (UID: "ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 10:20:30.927385 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:30.927302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:30.927527 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:30.927302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:30.927527 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.927418 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6z7nq" podUID="c0b6c7bc-5908-4525-aacc-0b8c3186c646" Jul 15 10:20:30.927527 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:30.927312 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:30.927687 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.927523 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdhc5" podUID="aa664fba-4556-458c-9496-dc4cd0338f13" Jul 15 10:20:30.927687 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:30.927568 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zgjkv" podUID="ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d" Jul 15 10:20:31.883227 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.883162 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-114.ec2.internal" event="NodeReady" Jul 15 10:20:31.883641 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.883257 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 10:20:31.924816 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.924789 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bf4bcd5c9-drqg5"] Jul 15 10:20:31.944967 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.944263 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4"] Jul 15 10:20:31.944967 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.944362 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:31.950662 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.946741 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qn4gb\"" Jul 15 10:20:31.950662 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.949543 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 10:20:31.950662 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.949731 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 10:20:31.950662 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.949934 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 10:20:31.955558 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.954813 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-6nkd9"] Jul 15 10:20:31.957828 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.957477 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:31.962716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.961076 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 10:20:31.962716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.961692 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 10:20:31.964481 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.964445 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 10:20:31.964742 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.964625 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 10:20:31.964742 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.964680 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-slrfp\"" Jul 15 10:20:31.967668 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.966818 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb"] Jul 15 10:20:31.967668 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.966960 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:31.970573 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.970107 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 15 10:20:31.970573 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.970144 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xwwvt\"" Jul 15 10:20:31.970573 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.970228 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 10:20:31.970573 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.970379 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 15 10:20:31.970573 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.970448 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 10:20:31.975154 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.975128 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 10:20:31.975386 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.975370 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 15 10:20:31.981215 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.981197 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d6976c5c8-mk6gv"] Jul 15 10:20:31.981304 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.981297 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb" Jul 15 10:20:31.983402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.983384 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 10:20:31.983484 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.983384 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8nc7q\"" Jul 15 10:20:31.983484 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:31.983427 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 10:20:32.001265 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.001248 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk"] Jul 15 10:20:32.001370 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.001356 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.003668 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.003651 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 10:20:32.003852 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.003838 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 10:20:32.003928 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.003875 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 10:20:32.003928 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.003880 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-q7jrv\"" Jul 15 10:20:32.004025 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.003967 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 10:20:32.004025 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.003970 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 10:20:32.004228 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.004216 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 10:20:32.017776 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.017759 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-bcmt7"] Jul 15 10:20:32.017881 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.017867 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:32.020546 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.020519 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 10:20:32.020712 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.020694 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xknbw\"" Jul 15 10:20:32.020789 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.020773 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 10:20:32.020844 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.020785 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 10:20:32.029985 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.029970 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj"] Jul 15 10:20:32.030093 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.030078 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.032855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.032837 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hp6zf\"" Jul 15 10:20:32.032956 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.032867 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 10:20:32.032956 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.032869 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 10:20:32.033482 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.033460 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 10:20:32.033576 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.033489 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 10:20:32.037742 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.037721 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 10:20:32.042394 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.042379 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4"] Jul 15 10:20:32.042467 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.042404 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bf4bcd5c9-drqg5"] Jul 15 10:20:32.042467 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.042418 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kxkx5"] Jul 15 10:20:32.042544 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.042487 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.044463 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.044446 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 10:20:32.044554 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.044451 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-t78l5\"" Jul 15 10:20:32.044695 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.044682 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 10:20:32.044770 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.044700 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 10:20:32.044770 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.044708 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 10:20:32.058606 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.058591 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p7mwj"] Jul 15 10:20:32.058710 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.058698 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.060841 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.060824 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jf9sn\"" Jul 15 10:20:32.060914 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.060828 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 10:20:32.060914 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.060871 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 10:20:32.069028 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.069012 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-6nkd9"] Jul 15 10:20:32.069126 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.069038 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl"] Jul 15 10:20:32.069166 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.069127 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7mwj" Jul 15 10:20:32.071235 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.071221 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 10:20:32.071321 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.071241 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 10:20:32.071452 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.071438 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vwrs8\"" Jul 15 10:20:32.071452 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.071445 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 10:20:32.081174 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.081155 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm"] Jul 15 10:20:32.081288 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.081274 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl" Jul 15 10:20:32.084155 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.084133 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 10:20:32.084242 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.084169 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pqnls\"" Jul 15 10:20:32.084413 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.084394 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 10:20:32.093634 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093621 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk"] Jul 15 10:20:32.093705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093647 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-bcmt7"] Jul 15 10:20:32.093705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093661 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kxkx5"] Jul 15 10:20:32.093705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093677 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl"] Jul 15 10:20:32.093705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093687 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb"] Jul 15 10:20:32.093705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093699 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7mwj"] Jul 15 10:20:32.093842 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093710 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj"] Jul 15 10:20:32.093842 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093717 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.093842 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093725 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d6976c5c8-mk6gv"] Jul 15 10:20:32.093842 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.093736 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm"] Jul 15 10:20:32.095721 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.095699 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 10:20:32.095813 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.095765 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 10:20:32.095813 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.095779 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 10:20:32.095896 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.095876 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ksvr7\"" Jul 15 10:20:32.096152 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.096140 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 10:20:32.097488 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097468 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57hpg\" (UniqueName: \"kubernetes.io/projected/b0621a97-3a04-46fd-af62-3a2af5065073-kube-api-access-57hpg\") pod \"volume-data-source-validator-78495597f6-9mmjb\" (UID: \"b0621a97-3a04-46fd-af62-3a2af5065073\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb" Jul 15 10:20:32.097585 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f10a8b3c-1033-4a14-a886-7b3a1503a92f-ca-trust-extracted\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.097585 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097543 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2eab90-c6a8-4554-b303-102ede8d8570-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.097585 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097565 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-certificates\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.097763 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097598 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.097763 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097640 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.097763 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097724 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.097763 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097755 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:32.097954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097782 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5d2eab90-c6a8-4554-b303-102ede8d8570-snapshots\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.097954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097806 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-stats-auth\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.097954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097864 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzhkx\" (UniqueName: \"kubernetes.io/projected/5d2eab90-c6a8-4554-b303-102ede8d8570-kube-api-access-wzhkx\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.097954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097888 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.097954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2xnz\" (UniqueName: \"kubernetes.io/projected/a2ccad75-c598-469a-adb8-10b2cb51c8e9-kube-api-access-x2xnz\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.097954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097937 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jnp4\" (UniqueName: \"kubernetes.io/projected/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-kube-api-access-4jnp4\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.098257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.097963 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-image-registry-private-configuration\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.098257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098004 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-installation-pull-secrets\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.098257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098094 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-bound-sa-token\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.098257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098120 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d2eab90-c6a8-4554-b303-102ede8d8570-tmp\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.098257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098149 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt6pg\" (UniqueName: \"kubernetes.io/projected/b941075e-602c-47fe-b101-c38edae9c86c-kube-api-access-qt6pg\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:32.098257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098185 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-trusted-ca\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.098257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098212 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2eab90-c6a8-4554-b303-102ede8d8570-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.098257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098248 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d2eab90-c6a8-4554-b303-102ede8d8570-serving-cert\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.098568 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098270 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-default-certificate\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.098568 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098294 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.098568 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.098338 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg5gm\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-kube-api-access-zg5gm\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.198779 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.198756 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/481a73d5-b917-4f18-84eb-03aa60dc1e1a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.198868 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.198788 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7421f994-e37a-4cf5-830c-70a5fabc555d-trusted-ca\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.198868 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.198806 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7421f994-e37a-4cf5-830c-70a5fabc555d-config\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.198868 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.198825 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5d2eab90-c6a8-4554-b303-102ede8d8570-snapshots\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.198868 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.198843 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.199078 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.198906 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-image-registry-private-configuration\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.199078 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.198939 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 10:20:32.199078 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.198944 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvpc5\" (UniqueName: \"kubernetes.io/projected/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-kube-api-access-tvpc5\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.199078 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.198982 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2xnz\" (UniqueName: \"kubernetes.io/projected/a2ccad75-c598-469a-adb8-10b2cb51c8e9-kube-api-access-x2xnz\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.199078 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.198993 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:32.69897708 +0000 UTC m=+36.214969544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : secret "router-metrics-certs-default" not found Jul 15 10:20:32.199078 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199017 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-installation-pull-secrets\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.199078 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199037 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-bound-sa-token\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.199078 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199076 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrpr9\" (UniqueName: \"kubernetes.io/projected/481a73d5-b917-4f18-84eb-03aa60dc1e1a-kube-api-access-wrpr9\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199099 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7421f994-e37a-4cf5-830c-70a5fabc555d-serving-cert\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199122 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d2eab90-c6a8-4554-b303-102ede8d8570-tmp\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199150 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6pg\" (UniqueName: \"kubernetes.io/projected/b941075e-602c-47fe-b101-c38edae9c86c-kube-api-access-qt6pg\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199175 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-default-certificate\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199216 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57hpg\" (UniqueName: \"kubernetes.io/projected/b0621a97-3a04-46fd-af62-3a2af5065073-kube-api-access-57hpg\") pod \"volume-data-source-validator-78495597f6-9mmjb\" (UID: \"b0621a97-3a04-46fd-af62-3a2af5065073\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f10a8b3c-1033-4a14-a886-7b3a1503a92f-ca-trust-extracted\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199283 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-tmp-dir\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199307 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199326 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/481a73d5-b917-4f18-84eb-03aa60dc1e1a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199356 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199375 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9p79\" (UniqueName: \"kubernetes.io/projected/323de29c-ec0b-42a7-a29f-e3e3a57fa074-kube-api-access-v9p79\") pod \"network-check-source-869dcfcdcd-kl8kl\" (UID: \"323de29c-ec0b-42a7-a29f-e3e3a57fa074\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199398 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-stats-auth\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199429 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199446 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5d2eab90-c6a8-4554-b303-102ede8d8570-snapshots\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.199468 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199454 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:32.200103 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199516 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jnp4\" (UniqueName: \"kubernetes.io/projected/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-kube-api-access-4jnp4\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.200103 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.199530 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 10:20:32.200103 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.199543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2eab90-c6a8-4554-b303-102ede8d8570-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.200103 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.199572 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls podName:b941075e-602c-47fe-b101-c38edae9c86c nodeName:}" failed. No retries permitted until 2026-07-15 10:20:32.699557945 +0000 UTC m=+36.215550423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls") pod "cluster-samples-operator-65db858894-7bgmk" (UID: "b941075e-602c-47fe-b101-c38edae9c86c") : secret "samples-operator-tls" not found Jul 15 10:20:32.200313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200134 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d2eab90-c6a8-4554-b303-102ede8d8570-tmp\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.200313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200186 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzhkx\" (UniqueName: \"kubernetes.io/projected/5d2eab90-c6a8-4554-b303-102ede8d8570-kube-api-access-wzhkx\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.200313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200222 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-config\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.200313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200246 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-trusted-ca\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.200313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200266 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p2jq\" (UniqueName: \"kubernetes.io/projected/7421f994-e37a-4cf5-830c-70a5fabc555d-kube-api-access-2p2jq\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.200313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200283 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d2eab90-c6a8-4554-b303-102ede8d8570-serving-cert\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.200313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200297 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200317 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmwj5\" (UniqueName: \"kubernetes.io/projected/e0e76093-626a-4493-a99e-e4ae77681681-kube-api-access-nmwj5\") pod \"ingress-canary-p7mwj\" (UID: \"e0e76093-626a-4493-a99e-e4ae77681681\") " pod="openshift-ingress-canary/ingress-canary-p7mwj" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200341 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl89c\" (UniqueName: \"kubernetes.io/projected/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-kube-api-access-gl89c\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200370 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg5gm\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-kube-api-access-zg5gm\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200395 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-serving-cert\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200417 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0e76093-626a-4493-a99e-e4ae77681681-cert\") pod \"ingress-canary-p7mwj\" (UID: \"e0e76093-626a-4493-a99e-e4ae77681681\") " pod="openshift-ingress-canary/ingress-canary-p7mwj" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200437 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-config-volume\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200464 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2eab90-c6a8-4554-b303-102ede8d8570-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200487 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.200589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200514 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-certificates\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.200951 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.200625 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2eab90-c6a8-4554-b303-102ede8d8570-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.200951 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.200779 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:32.700762119 +0000 UTC m=+36.216754586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : configmap references non-existent config key: service-ca.crt Jul 15 10:20:32.200951 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.200807 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 10:20:32.200951 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.200819 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bf4bcd5c9-drqg5: secret "image-registry-tls" not found Jul 15 10:20:32.200951 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.200866 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls podName:f10a8b3c-1033-4a14-a886-7b3a1503a92f nodeName:}" failed. No retries permitted until 2026-07-15 10:20:32.700853853 +0000 UTC m=+36.216846332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls") pod "image-registry-5bf4bcd5c9-drqg5" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f") : secret "image-registry-tls" not found Jul 15 10:20:32.201712 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.201276 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:32.201712 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.201336 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls podName:b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:32.701320869 +0000 UTC m=+36.217313332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hszc4" (UID: "b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581") : secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:32.201712 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.201666 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f10a8b3c-1033-4a14-a886-7b3a1503a92f-ca-trust-extracted\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.201891 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.201743 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.201891 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.201793 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-trusted-ca\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.202262 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.202240 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2eab90-c6a8-4554-b303-102ede8d8570-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.202547 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.202520 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-certificates\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.204481 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.204335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-image-registry-private-configuration\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.204481 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.204338 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-installation-pull-secrets\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.204481 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.204361 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-default-certificate\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.204481 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.204371 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d2eab90-c6a8-4554-b303-102ede8d8570-serving-cert\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.204481 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.204418 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-stats-auth\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.212982 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.212938 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2xnz\" (UniqueName: \"kubernetes.io/projected/a2ccad75-c598-469a-adb8-10b2cb51c8e9-kube-api-access-x2xnz\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.213334 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.213307 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-bound-sa-token\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.213674 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.213656 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6pg\" (UniqueName: \"kubernetes.io/projected/b941075e-602c-47fe-b101-c38edae9c86c-kube-api-access-qt6pg\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:32.213954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.213937 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzhkx\" (UniqueName: \"kubernetes.io/projected/5d2eab90-c6a8-4554-b303-102ede8d8570-kube-api-access-wzhkx\") pod \"insights-operator-68d9dd94cf-6nkd9\" (UID: \"5d2eab90-c6a8-4554-b303-102ede8d8570\") " pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.214041 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.214024 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57hpg\" (UniqueName: \"kubernetes.io/projected/b0621a97-3a04-46fd-af62-3a2af5065073-kube-api-access-57hpg\") pod \"volume-data-source-validator-78495597f6-9mmjb\" (UID: \"b0621a97-3a04-46fd-af62-3a2af5065073\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb" Jul 15 10:20:32.214259 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.214245 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg5gm\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-kube-api-access-zg5gm\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.214679 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.214665 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jnp4\" (UniqueName: \"kubernetes.io/projected/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-kube-api-access-4jnp4\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.282511 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.282487 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" Jul 15 10:20:32.288644 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.288622 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb" Jul 15 10:20:32.301100 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301082 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmwj5\" (UniqueName: \"kubernetes.io/projected/e0e76093-626a-4493-a99e-e4ae77681681-kube-api-access-nmwj5\") pod \"ingress-canary-p7mwj\" (UID: \"e0e76093-626a-4493-a99e-e4ae77681681\") " pod="openshift-ingress-canary/ingress-canary-p7mwj" Jul 15 10:20:32.301187 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301113 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl89c\" (UniqueName: \"kubernetes.io/projected/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-kube-api-access-gl89c\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.301187 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301140 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-serving-cert\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.301343 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301319 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0e76093-626a-4493-a99e-e4ae77681681-cert\") pod \"ingress-canary-p7mwj\" (UID: \"e0e76093-626a-4493-a99e-e4ae77681681\") " pod="openshift-ingress-canary/ingress-canary-p7mwj" Jul 15 10:20:32.301400 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301357 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-config-volume\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.301400 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301388 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.301563 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301417 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/481a73d5-b917-4f18-84eb-03aa60dc1e1a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.301563 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301455 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7421f994-e37a-4cf5-830c-70a5fabc555d-trusted-ca\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.301563 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.301478 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 10:20:32.301563 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7421f994-e37a-4cf5-830c-70a5fabc555d-config\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.301563 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.301545 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls podName:ae01fde5-5868-4c83-9b4d-9b8e52cfd453 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:32.801526041 +0000 UTC m=+36.317518511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls") pod "dns-default-kxkx5" (UID: "ae01fde5-5868-4c83-9b4d-9b8e52cfd453") : secret "dns-default-metrics-tls" not found Jul 15 10:20:32.301808 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301569 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpc5\" (UniqueName: \"kubernetes.io/projected/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-kube-api-access-tvpc5\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.301808 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301608 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrpr9\" (UniqueName: \"kubernetes.io/projected/481a73d5-b917-4f18-84eb-03aa60dc1e1a-kube-api-access-wrpr9\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.301808 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301635 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7421f994-e37a-4cf5-830c-70a5fabc555d-serving-cert\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.301808 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301697 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-tmp-dir\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.301808 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301738 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/481a73d5-b917-4f18-84eb-03aa60dc1e1a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.301808 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301767 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9p79\" (UniqueName: \"kubernetes.io/projected/323de29c-ec0b-42a7-a29f-e3e3a57fa074-kube-api-access-v9p79\") pod \"network-check-source-869dcfcdcd-kl8kl\" (UID: \"323de29c-ec0b-42a7-a29f-e3e3a57fa074\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl" Jul 15 10:20:32.302113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301865 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-config\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.302113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p2jq\" (UniqueName: \"kubernetes.io/projected/7421f994-e37a-4cf5-830c-70a5fabc555d-kube-api-access-2p2jq\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.302113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.301995 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-config-volume\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.302262 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.302146 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7421f994-e37a-4cf5-830c-70a5fabc555d-config\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.302411 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.302387 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-tmp-dir\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.302534 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.302495 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7421f994-e37a-4cf5-830c-70a5fabc555d-trusted-ca\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.302899 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.302878 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-config\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.302962 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.302941 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/481a73d5-b917-4f18-84eb-03aa60dc1e1a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.303494 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.303466 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-serving-cert\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.304147 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.304129 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0e76093-626a-4493-a99e-e4ae77681681-cert\") pod \"ingress-canary-p7mwj\" (UID: \"e0e76093-626a-4493-a99e-e4ae77681681\") " pod="openshift-ingress-canary/ingress-canary-p7mwj" Jul 15 10:20:32.304378 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.304355 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7421f994-e37a-4cf5-830c-70a5fabc555d-serving-cert\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.304676 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.304658 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/481a73d5-b917-4f18-84eb-03aa60dc1e1a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.310806 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.310730 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmwj5\" (UniqueName: \"kubernetes.io/projected/e0e76093-626a-4493-a99e-e4ae77681681-kube-api-access-nmwj5\") pod \"ingress-canary-p7mwj\" (UID: \"e0e76093-626a-4493-a99e-e4ae77681681\") " pod="openshift-ingress-canary/ingress-canary-p7mwj" Jul 15 10:20:32.312383 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.312362 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl89c\" (UniqueName: \"kubernetes.io/projected/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-kube-api-access-gl89c\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.315689 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.315666 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpc5\" (UniqueName: \"kubernetes.io/projected/8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6-kube-api-access-tvpc5\") pod \"service-ca-operator-7695fc6c8c-2m7mm\" (UID: \"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.315865 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.315842 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrpr9\" (UniqueName: \"kubernetes.io/projected/481a73d5-b917-4f18-84eb-03aa60dc1e1a-kube-api-access-wrpr9\") pod \"kube-storage-version-migrator-operator-799cb7f958-t4dcj\" (UID: \"481a73d5-b917-4f18-84eb-03aa60dc1e1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.316342 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.316324 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9p79\" (UniqueName: \"kubernetes.io/projected/323de29c-ec0b-42a7-a29f-e3e3a57fa074-kube-api-access-v9p79\") pod \"network-check-source-869dcfcdcd-kl8kl\" (UID: \"323de29c-ec0b-42a7-a29f-e3e3a57fa074\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl" Jul 15 10:20:32.325875 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.325721 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p2jq\" (UniqueName: \"kubernetes.io/projected/7421f994-e37a-4cf5-830c-70a5fabc555d-kube-api-access-2p2jq\") pod \"console-operator-7866497494-bcmt7\" (UID: \"7421f994-e37a-4cf5-830c-70a5fabc555d\") " pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.339259 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.339159 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:32.350510 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.350486 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" Jul 15 10:20:32.376630 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.376437 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7mwj" Jul 15 10:20:32.389581 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.389557 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl" Jul 15 10:20:32.402582 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.402496 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" Jul 15 10:20:32.480470 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.480445 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-6nkd9"] Jul 15 10:20:32.483645 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.483618 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb"] Jul 15 10:20:32.640735 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.640710 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj"] Jul 15 10:20:32.644810 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.644789 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm"] Jul 15 10:20:32.654174 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.654155 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl"] Jul 15 10:20:32.654898 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.654882 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7mwj"] Jul 15 10:20:32.656436 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.656418 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-bcmt7"] Jul 15 10:20:32.659587 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:20:32.659560 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7421f994_e37a_4cf5_830c_70a5fabc555d.slice/crio-2ba5a09a9bc0679489607e40265181e4c47985ce1c37108a872a7a3016073c96 WatchSource:0}: Error finding container 2ba5a09a9bc0679489607e40265181e4c47985ce1c37108a872a7a3016073c96: Status 404 returned error can't find the container with id 2ba5a09a9bc0679489607e40265181e4c47985ce1c37108a872a7a3016073c96 Jul 15 10:20:32.660043 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:20:32.660015 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eac6d3c_3a8b_4fc0_92ef_b6ca6def64a6.slice/crio-2508fe18e8e3b6c2c84bd016e4c364633ce6e8744182165c49623b2559911066 WatchSource:0}: Error finding container 2508fe18e8e3b6c2c84bd016e4c364633ce6e8744182165c49623b2559911066: Status 404 returned error can't find the container with id 2508fe18e8e3b6c2c84bd016e4c364633ce6e8744182165c49623b2559911066 Jul 15 10:20:32.661117 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:20:32.660845 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e76093_626a_4493_a99e_e4ae77681681.slice/crio-1fe0663fea9f6c324c114dfecb26a3af43f389b7c5cc26f5ab7e690b443b2da1 WatchSource:0}: Error finding container 1fe0663fea9f6c324c114dfecb26a3af43f389b7c5cc26f5ab7e690b443b2da1: Status 404 returned error can't find the container with id 1fe0663fea9f6c324c114dfecb26a3af43f389b7c5cc26f5ab7e690b443b2da1 Jul 15 10:20:32.661995 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:20:32.661792 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323de29c_ec0b_42a7_a29f_e3e3a57fa074.slice/crio-0333a6a412c605d0105568d112f9bc119e3a6968278df3bab677159cb414ef63 WatchSource:0}: Error finding container 0333a6a412c605d0105568d112f9bc119e3a6968278df3bab677159cb414ef63: Status 404 returned error can't find the container with id 0333a6a412c605d0105568d112f9bc119e3a6968278df3bab677159cb414ef63 Jul 15 10:20:32.663165 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:20:32.663128 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481a73d5_b917_4f18_84eb_03aa60dc1e1a.slice/crio-b1daf44ae7565d8438cfc52b984f88c0b5669779026d54290443e5d4c5805b1a WatchSource:0}: Error finding container b1daf44ae7565d8438cfc52b984f88c0b5669779026d54290443e5d4c5805b1a: Status 404 returned error can't find the container with id b1daf44ae7565d8438cfc52b984f88c0b5669779026d54290443e5d4c5805b1a Jul 15 10:20:32.705873 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.705853 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:32.705975 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.705888 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:32.705975 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.705908 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:32.705975 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.705963 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706002 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706015 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bf4bcd5c9-drqg5: secret "image-registry-tls" not found Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.706019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706022 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706081 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls podName:f10a8b3c-1033-4a14-a886-7b3a1503a92f nodeName:}" failed. No retries permitted until 2026-07-15 10:20:33.706042245 +0000 UTC m=+37.222034722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls") pod "image-registry-5bf4bcd5c9-drqg5" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f") : secret "image-registry-tls" not found Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706086 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706101 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls podName:b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:33.70608961 +0000 UTC m=+37.222082076 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hszc4" (UID: "b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581") : secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706136 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls podName:b941075e-602c-47fe-b101-c38edae9c86c nodeName:}" failed. No retries permitted until 2026-07-15 10:20:33.706122662 +0000 UTC m=+37.222115129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls") pod "cluster-samples-operator-65db858894-7bgmk" (UID: "b941075e-602c-47fe-b101-c38edae9c86c") : secret "samples-operator-tls" not found Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706143 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 10:20:32.706148 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706149 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:33.706143457 +0000 UTC m=+37.222135920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : configmap references non-existent config key: service-ca.crt Jul 15 10:20:32.706514 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.706169 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:33.706162495 +0000 UTC m=+37.222154958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : secret "router-metrics-certs-default" not found Jul 15 10:20:32.806918 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.806895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:32.807019 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.807005 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 10:20:32.807085 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:32.807045 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls podName:ae01fde5-5868-4c83-9b4d-9b8e52cfd453 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:33.807034789 +0000 UTC m=+37.323027252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls") pod "dns-default-kxkx5" (UID: "ae01fde5-5868-4c83-9b4d-9b8e52cfd453") : secret "dns-default-metrics-tls" not found Jul 15 10:20:32.927600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.927549 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:20:32.928108 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.927552 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:20:32.928108 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.927878 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:20:32.932452 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.930726 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 10:20:32.932452 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.930944 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 10:20:32.932452 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.931010 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-krhcr\"" Jul 15 10:20:32.932452 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:32.931243 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fdfwf\"" Jul 15 10:20:33.173077 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.173044 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl" event={"ID":"323de29c-ec0b-42a7-a29f-e3e3a57fa074","Type":"ContainerStarted","Data":"0333a6a412c605d0105568d112f9bc119e3a6968278df3bab677159cb414ef63"} Jul 15 10:20:33.173995 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.173969 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" event={"ID":"5d2eab90-c6a8-4554-b303-102ede8d8570","Type":"ContainerStarted","Data":"721c1f7f517e69c5d8de2dc122d17be7d4366b0da628c915dcfca4e6aeb1eac3"} Jul 15 10:20:33.174893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.174875 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-bcmt7" event={"ID":"7421f994-e37a-4cf5-830c-70a5fabc555d","Type":"ContainerStarted","Data":"2ba5a09a9bc0679489607e40265181e4c47985ce1c37108a872a7a3016073c96"} Jul 15 10:20:33.175927 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.175909 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7mwj" event={"ID":"e0e76093-626a-4493-a99e-e4ae77681681","Type":"ContainerStarted","Data":"1fe0663fea9f6c324c114dfecb26a3af43f389b7c5cc26f5ab7e690b443b2da1"} Jul 15 10:20:33.178898 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.178853 2585 generic.go:356] "Generic (PLEG): container finished" podID="7638a8fc-3b2c-4493-b111-a1e1fdac574c" containerID="757f6c9b4afcdb105db23a50226969a4def4bc1ea0a3847c74a0cc18069ce889" exitCode=0 Jul 15 10:20:33.178971 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.178923 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" event={"ID":"7638a8fc-3b2c-4493-b111-a1e1fdac574c","Type":"ContainerDied","Data":"757f6c9b4afcdb105db23a50226969a4def4bc1ea0a3847c74a0cc18069ce889"} Jul 15 10:20:33.180035 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.180017 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" event={"ID":"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6","Type":"ContainerStarted","Data":"2508fe18e8e3b6c2c84bd016e4c364633ce6e8744182165c49623b2559911066"} Jul 15 10:20:33.181042 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.181016 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" event={"ID":"481a73d5-b917-4f18-84eb-03aa60dc1e1a","Type":"ContainerStarted","Data":"b1daf44ae7565d8438cfc52b984f88c0b5669779026d54290443e5d4c5805b1a"} Jul 15 10:20:33.181977 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.181959 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb" event={"ID":"b0621a97-3a04-46fd-af62-3a2af5065073","Type":"ContainerStarted","Data":"0728df7a4166defa94a274811058f38e67e29c0eeda3a3e6cfdcb2e54f27bd31"} Jul 15 10:20:33.715564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.715364 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:33.715770 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.715586 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:33.715770 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.715617 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:33.715770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715538 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 10:20:33.715770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715650 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bf4bcd5c9-drqg5: secret "image-registry-tls" not found Jul 15 10:20:33.715770 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.715680 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:33.715770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715713 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls podName:f10a8b3c-1033-4a14-a886-7b3a1503a92f nodeName:}" failed. No retries permitted until 2026-07-15 10:20:35.71569386 +0000 UTC m=+39.231686363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls") pod "image-registry-5bf4bcd5c9-drqg5" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f") : secret "image-registry-tls" not found Jul 15 10:20:33.715770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715758 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:33.715770 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715771 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 10:20:33.716192 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.715787 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:33.716192 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715805 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls podName:b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:35.715793543 +0000 UTC m=+39.231786010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hszc4" (UID: "b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581") : secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:33.716192 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715835 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls podName:b941075e-602c-47fe-b101-c38edae9c86c nodeName:}" failed. No retries permitted until 2026-07-15 10:20:35.71582503 +0000 UTC m=+39.231817505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls") pod "cluster-samples-operator-65db858894-7bgmk" (UID: "b941075e-602c-47fe-b101-c38edae9c86c") : secret "samples-operator-tls" not found Jul 15 10:20:33.716192 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715845 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 10:20:33.716192 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715861 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:35.71584805 +0000 UTC m=+39.231840533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : configmap references non-existent config key: service-ca.crt Jul 15 10:20:33.716192 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.715881 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:35.715872001 +0000 UTC m=+39.231864468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : secret "router-metrics-certs-default" not found Jul 15 10:20:33.816363 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:33.816333 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:33.816634 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.816547 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 10:20:33.816634 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:33.816625 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls podName:ae01fde5-5868-4c83-9b4d-9b8e52cfd453 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:35.816606745 +0000 UTC m=+39.332599223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls") pod "dns-default-kxkx5" (UID: "ae01fde5-5868-4c83-9b4d-9b8e52cfd453") : secret "dns-default-metrics-tls" not found Jul 15 10:20:34.191666 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:34.190981 2585 generic.go:356] "Generic (PLEG): container finished" podID="7638a8fc-3b2c-4493-b111-a1e1fdac574c" containerID="f051932f011e0d185d1338d9161180d91a0dda90a383e57d8f24fab5b87e0533" exitCode=0 Jul 15 10:20:34.191666 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:34.191237 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" event={"ID":"7638a8fc-3b2c-4493-b111-a1e1fdac574c","Type":"ContainerDied","Data":"f051932f011e0d185d1338d9161180d91a0dda90a383e57d8f24fab5b87e0533"} Jul 15 10:20:35.199032 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.198907 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" event={"ID":"7638a8fc-3b2c-4493-b111-a1e1fdac574c","Type":"ContainerStarted","Data":"d97dbf2885a7e106d5354dc978d16ed27c582b9ee8a6f2656de2cbd8215b0494"} Jul 15 10:20:35.201224 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.201170 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb" event={"ID":"b0621a97-3a04-46fd-af62-3a2af5065073","Type":"ContainerStarted","Data":"6d5a87324a74c253e023671683b39eae8210a6c164c7034f68ce6038daaf73be"} Jul 15 10:20:35.228449 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.228402 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dmsg6" podStartSLOduration=8.201688575 podStartE2EDuration="38.228385353s" totalImagesPullingTime="30.026696778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:35.227176992 +0000 UTC m=+38.743169479" watchObservedRunningTime="2026-07-15 10:20:35.228385353 +0000 UTC m=+38.744377839" Jul 15 10:20:35.251862 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.251818 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9mmjb" podStartSLOduration=26.477213021 podStartE2EDuration="28.251801021s" totalImagesPullingTime="1.774588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:35.251183635 +0000 UTC m=+38.767176125" watchObservedRunningTime="2026-07-15 10:20:35.251801021 +0000 UTC m=+38.767793505" Jul 15 10:20:35.734161 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.734126 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:35.734344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.734196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:35.734344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.734258 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:35.734344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.734290 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:35.734344 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734309 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:39.734285924 +0000 UTC m=+43.250278407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : configmap references non-existent config key: service-ca.crt Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.734354 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734365 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734371 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734410 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:39.73439835 +0000 UTC m=+43.250390818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : secret "router-metrics-certs-default" not found Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734434 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls podName:b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:39.734416946 +0000 UTC m=+43.250409423 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hszc4" (UID: "b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581") : secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734462 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734469 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734480 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bf4bcd5c9-drqg5: secret "image-registry-tls" not found Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734495 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls podName:b941075e-602c-47fe-b101-c38edae9c86c nodeName:}" failed. No retries permitted until 2026-07-15 10:20:39.73448502 +0000 UTC m=+43.250477501 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls") pod "cluster-samples-operator-65db858894-7bgmk" (UID: "b941075e-602c-47fe-b101-c38edae9c86c") : secret "samples-operator-tls" not found Jul 15 10:20:35.734553 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.734510 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls podName:f10a8b3c-1033-4a14-a886-7b3a1503a92f nodeName:}" failed. No retries permitted until 2026-07-15 10:20:39.734500477 +0000 UTC m=+43.250492943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls") pod "image-registry-5bf4bcd5c9-drqg5" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f") : secret "image-registry-tls" not found Jul 15 10:20:35.834982 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:35.834946 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:35.835181 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.835147 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 10:20:35.835259 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:35.835228 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls podName:ae01fde5-5868-4c83-9b4d-9b8e52cfd453 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:39.835211275 +0000 UTC m=+43.351203756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls") pod "dns-default-kxkx5" (UID: "ae01fde5-5868-4c83-9b4d-9b8e52cfd453") : secret "dns-default-metrics-tls" not found Jul 15 10:20:39.771481 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:39.771457 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:39.771493 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:39.771510 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:39.771593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771608 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771611 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771659 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls podName:b941075e-602c-47fe-b101-c38edae9c86c nodeName:}" failed. No retries permitted until 2026-07-15 10:20:47.771641395 +0000 UTC m=+51.287633861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls") pod "cluster-samples-operator-65db858894-7bgmk" (UID: "b941075e-602c-47fe-b101-c38edae9c86c") : secret "samples-operator-tls" not found Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771670 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771678 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bf4bcd5c9-drqg5: secret "image-registry-tls" not found Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771678 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls podName:b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:47.771670075 +0000 UTC m=+51.287662540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hszc4" (UID: "b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581") : secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771737 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls podName:f10a8b3c-1033-4a14-a886-7b3a1503a92f nodeName:}" failed. No retries permitted until 2026-07-15 10:20:47.771706467 +0000 UTC m=+51.287698946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls") pod "image-registry-5bf4bcd5c9-drqg5" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f") : secret "image-registry-tls" not found Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771758 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:47.77174861 +0000 UTC m=+51.287741074 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : configmap references non-existent config key: service-ca.crt Jul 15 10:20:39.771895 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:39.771806 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:39.772320 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771942 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 10:20:39.772320 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.771992 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:47.771979282 +0000 UTC m=+51.287971762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : secret "router-metrics-certs-default" not found Jul 15 10:20:39.872439 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:39.872322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:39.872552 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.872454 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 10:20:39.872552 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:39.872518 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls podName:ae01fde5-5868-4c83-9b4d-9b8e52cfd453 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:47.872499138 +0000 UTC m=+51.388491605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls") pod "dns-default-kxkx5" (UID: "ae01fde5-5868-4c83-9b4d-9b8e52cfd453") : secret "dns-default-metrics-tls" not found Jul 15 10:20:40.216307 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.216251 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" event={"ID":"5d2eab90-c6a8-4554-b303-102ede8d8570","Type":"ContainerStarted","Data":"3e3064a5ba29a3a3dfce03e9e8f85d580532e97d2517a8c51d7c7e156ff08228"} Jul 15 10:20:40.218345 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.218322 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/0.log" Jul 15 10:20:40.218448 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.218369 2585 generic.go:356] "Generic (PLEG): container finished" podID="7421f994-e37a-4cf5-830c-70a5fabc555d" containerID="f9b12191a048326e02a9f6f15745be6ab72d4f4966507bcfa09cc8b2af18b4ac" exitCode=255 Jul 15 10:20:40.218497 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.218448 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-bcmt7" event={"ID":"7421f994-e37a-4cf5-830c-70a5fabc555d","Type":"ContainerDied","Data":"f9b12191a048326e02a9f6f15745be6ab72d4f4966507bcfa09cc8b2af18b4ac"} Jul 15 10:20:40.218949 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.218876 2585 scope.go:117] "RemoveContainer" containerID="f9b12191a048326e02a9f6f15745be6ab72d4f4966507bcfa09cc8b2af18b4ac" Jul 15 10:20:40.220116 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.220082 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7mwj" event={"ID":"e0e76093-626a-4493-a99e-e4ae77681681","Type":"ContainerStarted","Data":"230cb896a799fad1e361be460e0f352a61ff26ddb037b051f06cc5eb09f7f703"} Jul 15 10:20:40.222266 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.222098 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" event={"ID":"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6","Type":"ContainerStarted","Data":"ad75f1ea820a66bc6744fad68756c0a3e94cee900bbc39c6c71e78c727c7f0ff"} Jul 15 10:20:40.223621 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.223600 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl" event={"ID":"323de29c-ec0b-42a7-a29f-e3e3a57fa074","Type":"ContainerStarted","Data":"b7400658a8e6a8c74784d32686f80a3005c9b850bf584e4873d1e544ff6a1da0"} Jul 15 10:20:40.262768 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.262469 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" podStartSLOduration=29.085584172 podStartE2EDuration="36.262448665s" totalImagesPullingTime="7.176864493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:40.239687807 +0000 UTC m=+43.755680294" watchObservedRunningTime="2026-07-15 10:20:40.262448665 +0000 UTC m=+43.778441152" Jul 15 10:20:40.263418 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.263257 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p7mwj" podStartSLOduration=2.256152105 podStartE2EDuration="9.26324871s" totalImagesPullingTime="7.007096605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:40.261747487 +0000 UTC m=+43.777739973" watchObservedRunningTime="2026-07-15 10:20:40.26324871 +0000 UTC m=+43.779241177" Jul 15 10:20:40.324512 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.323744 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-kl8kl" podStartSLOduration=19.31714949 podStartE2EDuration="26.323727165s" totalImagesPullingTime="7.006577675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:40.283752982 +0000 UTC m=+43.799745467" watchObservedRunningTime="2026-07-15 10:20:40.323727165 +0000 UTC m=+43.839719651" Jul 15 10:20:40.324512 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.323994 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" podStartSLOduration=19.31600911 podStartE2EDuration="26.323984623s" totalImagesPullingTime="7.007975513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:40.312440915 +0000 UTC m=+43.828433411" watchObservedRunningTime="2026-07-15 10:20:40.323984623 +0000 UTC m=+43.839977109" Jul 15 10:20:40.836883 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.836852 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hzjrv"] Jul 15 10:20:40.869017 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.868990 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hzjrv"] Jul 15 10:20:40.869182 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.869141 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.872025 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.871994 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 10:20:40.872180 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.871994 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 10:20:40.872180 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.872175 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b6kmk\"" Jul 15 10:20:40.882385 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.882358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37035e8a-7489-4a0a-97fc-dbb3739cc852-crio-socket\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.882511 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.882398 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37035e8a-7489-4a0a-97fc-dbb3739cc852-data-volume\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.882511 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.882446 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.882655 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.882559 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.882655 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.882620 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrv4f\" (UniqueName: \"kubernetes.io/projected/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-api-access-zrv4f\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.983370 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.983303 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrv4f\" (UniqueName: \"kubernetes.io/projected/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-api-access-zrv4f\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.983370 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.983352 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37035e8a-7489-4a0a-97fc-dbb3739cc852-crio-socket\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.983544 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.983376 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37035e8a-7489-4a0a-97fc-dbb3739cc852-data-volume\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.983544 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.983401 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.983544 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.983437 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37035e8a-7489-4a0a-97fc-dbb3739cc852-crio-socket\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.983544 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.983527 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.983544 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:40.983532 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 10:20:40.983766 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:40.983584 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:41.483568184 +0000 UTC m=+44.999560651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : secret "insights-runtime-extractor-tls" not found Jul 15 10:20:40.983766 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:40.983613 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 10:20:40.983766 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:40.983652 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:41.483641831 +0000 UTC m=+44.999634297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : configmap "kube-rbac-proxy" not found Jul 15 10:20:40.983766 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.983730 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37035e8a-7489-4a0a-97fc-dbb3739cc852-data-volume\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:40.995113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:40.995091 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrv4f\" (UniqueName: \"kubernetes.io/projected/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-api-access-zrv4f\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:41.227424 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.227399 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:20:41.227824 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.227806 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/0.log" Jul 15 10:20:41.227930 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.227846 2585 generic.go:356] "Generic (PLEG): container finished" podID="7421f994-e37a-4cf5-830c-70a5fabc555d" containerID="c0ec7186cc48c4672980dcf4064d5e2eea1222865c2059d6bbd099fb5235de66" exitCode=255 Jul 15 10:20:41.227983 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.227929 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-bcmt7" event={"ID":"7421f994-e37a-4cf5-830c-70a5fabc555d","Type":"ContainerDied","Data":"c0ec7186cc48c4672980dcf4064d5e2eea1222865c2059d6bbd099fb5235de66"} Jul 15 10:20:41.228033 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.227995 2585 scope.go:117] "RemoveContainer" containerID="f9b12191a048326e02a9f6f15745be6ab72d4f4966507bcfa09cc8b2af18b4ac" Jul 15 10:20:41.228225 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.228206 2585 scope.go:117] "RemoveContainer" containerID="c0ec7186cc48c4672980dcf4064d5e2eea1222865c2059d6bbd099fb5235de66" Jul 15 10:20:41.228423 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:41.228395 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-bcmt7_openshift-console-operator(7421f994-e37a-4cf5-830c-70a5fabc555d)\"" pod="openshift-console-operator/console-operator-7866497494-bcmt7" podUID="7421f994-e37a-4cf5-830c-70a5fabc555d" Jul 15 10:20:41.229783 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.229743 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" event={"ID":"481a73d5-b917-4f18-84eb-03aa60dc1e1a","Type":"ContainerStarted","Data":"e3fdc45334bf8318f114e9bd670420b31f9795d6d5bab3b1525981fb281452e0"} Jul 15 10:20:41.282249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.282212 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" podStartSLOduration=20.836614322 podStartE2EDuration="28.28220054s" totalImagesPullingTime="7.445586218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:41.281436134 +0000 UTC m=+44.797428619" watchObservedRunningTime="2026-07-15 10:20:41.28220054 +0000 UTC m=+44.798193024" Jul 15 10:20:41.488291 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.488260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:41.488455 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:41.488360 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:41.488455 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:41.488394 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 10:20:41.488455 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:41.488437 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 10:20:41.488455 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:41.488452 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:42.488436513 +0000 UTC m=+46.004428977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : secret "insights-runtime-extractor-tls" not found Jul 15 10:20:41.488655 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:41.488477 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:42.488466833 +0000 UTC m=+46.004459296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : configmap "kube-rbac-proxy" not found Jul 15 10:20:42.235469 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:42.235435 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:20:42.235997 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:42.235940 2585 scope.go:117] "RemoveContainer" containerID="c0ec7186cc48c4672980dcf4064d5e2eea1222865c2059d6bbd099fb5235de66" Jul 15 10:20:42.236178 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:42.236155 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-bcmt7_openshift-console-operator(7421f994-e37a-4cf5-830c-70a5fabc555d)\"" pod="openshift-console-operator/console-operator-7866497494-bcmt7" podUID="7421f994-e37a-4cf5-830c-70a5fabc555d" Jul 15 10:20:42.339850 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:42.339823 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:42.339974 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:42.339869 2585 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:42.495842 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:42.495785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:42.495940 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:42.495853 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:42.495940 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:42.495886 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 10:20:42.495940 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:42.495931 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:44.495918907 +0000 UTC m=+48.011911371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : configmap "kube-rbac-proxy" not found Jul 15 10:20:42.495940 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:42.495939 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 10:20:42.496125 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:42.495972 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:44.495962304 +0000 UTC m=+48.011954768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : secret "insights-runtime-extractor-tls" not found Jul 15 10:20:42.616894 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:42.616875 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-92kbz_d842f042-1ae5-4efe-a17a-7da9e8f76567/dns-node-resolver/0.log" Jul 15 10:20:43.237956 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:43.237932 2585 scope.go:117] "RemoveContainer" containerID="c0ec7186cc48c4672980dcf4064d5e2eea1222865c2059d6bbd099fb5235de66" Jul 15 10:20:43.238287 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:43.238084 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-bcmt7_openshift-console-operator(7421f994-e37a-4cf5-830c-70a5fabc555d)\"" pod="openshift-console-operator/console-operator-7866497494-bcmt7" podUID="7421f994-e37a-4cf5-830c-70a5fabc555d" Jul 15 10:20:43.616213 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:43.616157 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j4p58_dd8b7c91-b49c-45bb-ae5b-e5ebe251de69/node-ca/0.log" Jul 15 10:20:44.512320 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:44.512283 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:44.512746 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:44.512371 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:44.512746 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:44.512428 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 10:20:44.512746 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:44.512463 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 10:20:44.512746 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:44.512508 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:48.512492901 +0000 UTC m=+52.028485364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : secret "insights-runtime-extractor-tls" not found Jul 15 10:20:44.512746 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:44.512528 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:48.51251892 +0000 UTC m=+52.028511385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : configmap "kube-rbac-proxy" not found Jul 15 10:20:44.816473 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:44.816406 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7mwj_e0e76093-626a-4493-a99e-e4ae77681681/serve-healthcheck-canary/0.log" Jul 15 10:20:47.840212 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:47.840169 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:47.840264 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:47.840302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:47.840331 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:47.840412 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840330 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840382 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840384 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840594 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bf4bcd5c9-drqg5: secret "image-registry-tls" not found Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840421 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840515 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:21:03.84049676 +0000 UTC m=+67.356489224 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : secret "router-metrics-certs-default" not found Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840702 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle podName:a2ccad75-c598-469a-adb8-10b2cb51c8e9 nodeName:}" failed. No retries permitted until 2026-07-15 10:21:03.840683098 +0000 UTC m=+67.356675564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle") pod "router-default-7d6976c5c8-mk6gv" (UID: "a2ccad75-c598-469a-adb8-10b2cb51c8e9") : configmap references non-existent config key: service-ca.crt Jul 15 10:20:47.840715 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840720 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls podName:b941075e-602c-47fe-b101-c38edae9c86c nodeName:}" failed. No retries permitted until 2026-07-15 10:21:03.840708918 +0000 UTC m=+67.356701381 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls") pod "cluster-samples-operator-65db858894-7bgmk" (UID: "b941075e-602c-47fe-b101-c38edae9c86c") : secret "samples-operator-tls" not found Jul 15 10:20:47.841172 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840735 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls podName:f10a8b3c-1033-4a14-a886-7b3a1503a92f nodeName:}" failed. No retries permitted until 2026-07-15 10:21:03.840726756 +0000 UTC m=+67.356719225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls") pod "image-registry-5bf4bcd5c9-drqg5" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f") : secret "image-registry-tls" not found Jul 15 10:20:47.841172 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.840756 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls podName:b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581 nodeName:}" failed. No retries permitted until 2026-07-15 10:21:03.84074777 +0000 UTC m=+67.356740234 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hszc4" (UID: "b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581") : secret "cluster-monitoring-operator-tls" not found Jul 15 10:20:47.941554 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:47.941527 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:20:47.941702 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.941669 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 10:20:47.941771 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:47.941726 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls podName:ae01fde5-5868-4c83-9b4d-9b8e52cfd453 nodeName:}" failed. No retries permitted until 2026-07-15 10:21:03.941711266 +0000 UTC m=+67.457703753 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls") pod "dns-default-kxkx5" (UID: "ae01fde5-5868-4c83-9b4d-9b8e52cfd453") : secret "dns-default-metrics-tls" not found Jul 15 10:20:48.546268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:48.546240 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:48.546449 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:48.546352 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:48.546515 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:48.546468 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 10:20:48.546567 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:20:48.546531 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls podName:37035e8a-7489-4a0a-97fc-dbb3739cc852 nodeName:}" failed. No retries permitted until 2026-07-15 10:20:56.546514086 +0000 UTC m=+60.062506550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzjrv" (UID: "37035e8a-7489-4a0a-97fc-dbb3739cc852") : secret "insights-runtime-extractor-tls" not found Jul 15 10:20:48.546797 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:48.546766 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37035e8a-7489-4a0a-97fc-dbb3739cc852-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:56.168035 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:56.168009 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-94td4" Jul 15 10:20:56.612558 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:56.612519 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:56.614965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:56.614936 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37035e8a-7489-4a0a-97fc-dbb3739cc852-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzjrv\" (UID: \"37035e8a-7489-4a0a-97fc-dbb3739cc852\") " pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:56.780326 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:56.780292 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hzjrv" Jul 15 10:20:56.920881 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:56.920855 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hzjrv"] Jul 15 10:20:56.923666 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:20:56.923646 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37035e8a_7489_4a0a_97fc_dbb3739cc852.slice/crio-f68ec6bd5f215cde013910c964e25d3e818b0af2c0b7b196d85bb283ee3d71d1 WatchSource:0}: Error finding container f68ec6bd5f215cde013910c964e25d3e818b0af2c0b7b196d85bb283ee3d71d1: Status 404 returned error can't find the container with id f68ec6bd5f215cde013910c964e25d3e818b0af2c0b7b196d85bb283ee3d71d1 Jul 15 10:20:57.283577 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:57.283547 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hzjrv" event={"ID":"37035e8a-7489-4a0a-97fc-dbb3739cc852","Type":"ContainerStarted","Data":"f68ec6bd5f215cde013910c964e25d3e818b0af2c0b7b196d85bb283ee3d71d1"} Jul 15 10:20:57.927661 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:57.927628 2585 scope.go:117] "RemoveContainer" containerID="c0ec7186cc48c4672980dcf4064d5e2eea1222865c2059d6bbd099fb5235de66" Jul 15 10:20:58.289932 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:58.289898 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:20:58.290370 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:58.289999 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-bcmt7" event={"ID":"7421f994-e37a-4cf5-830c-70a5fabc555d","Type":"ContainerStarted","Data":"ce64e07c24d4b152e7b45dbc9449e5241d84380113bf51738d11519c787b7f27"} Jul 15 10:20:58.290370 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:58.290294 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:20:58.314958 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:58.314907 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-bcmt7" podStartSLOduration=42.306776158 podStartE2EDuration="49.314888624s" totalImagesPullingTime="7.008112466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:20:58.313553647 +0000 UTC m=+61.829546143" watchObservedRunningTime="2026-07-15 10:20:58.314888624 +0000 UTC m=+61.830881109" Jul 15 10:20:58.644241 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:20:58.644164 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-bcmt7" Jul 15 10:21:02.305120 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.305087 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hzjrv" event={"ID":"37035e8a-7489-4a0a-97fc-dbb3739cc852","Type":"ContainerStarted","Data":"3851028dd1bae01429f2dbdc62db66b8071ef2a28abb5efe7786405f23be48fd"} Jul 15 10:21:02.666142 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.666057 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:21:02.666142 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.666129 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:21:02.666345 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.666148 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:21:02.668536 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.668512 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnglt\" (UniqueName: \"kubernetes.io/projected/ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d-kube-api-access-wnglt\") pod \"network-check-target-zgjkv\" (UID: \"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d\") " pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:21:02.668805 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.668788 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 10:21:02.669556 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.669540 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 10:21:02.678449 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.678427 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b6c7bc-5908-4525-aacc-0b8c3186c646-original-pull-secret\") pod \"global-pull-secret-syncer-6z7nq\" (UID: \"c0b6c7bc-5908-4525-aacc-0b8c3186c646\") " pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:21:02.678527 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.678433 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa664fba-4556-458c-9496-dc4cd0338f13-metrics-certs\") pod \"network-metrics-daemon-pdhc5\" (UID: \"aa664fba-4556-458c-9496-dc4cd0338f13\") " pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:21:02.938894 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.938843 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6z7nq" Jul 15 10:21:02.947092 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.947057 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fdfwf\"" Jul 15 10:21:02.951299 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.951281 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-krhcr\"" Jul 15 10:21:02.954854 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.954839 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:21:02.960054 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:02.960038 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdhc5" Jul 15 10:21:03.115992 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.115953 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6z7nq"] Jul 15 10:21:03.118332 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:03.118290 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b6c7bc_5908_4525_aacc_0b8c3186c646.slice/crio-11edc478434dab86cbe550b17f159fe2aacf9fe79ebe09b7c49135d82c080f9d WatchSource:0}: Error finding container 11edc478434dab86cbe550b17f159fe2aacf9fe79ebe09b7c49135d82c080f9d: Status 404 returned error can't find the container with id 11edc478434dab86cbe550b17f159fe2aacf9fe79ebe09b7c49135d82c080f9d Jul 15 10:21:03.157475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.157097 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zgjkv"] Jul 15 10:21:03.158996 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:03.158973 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5fe3cb_b4c9_44b1_a2bf_b1eb746eb81d.slice/crio-eb66049a4b710139cb002e128f04fe0beb96d6cccba6e9e46ed7495270d28cbd WatchSource:0}: Error finding container eb66049a4b710139cb002e128f04fe0beb96d6cccba6e9e46ed7495270d28cbd: Status 404 returned error can't find the container with id eb66049a4b710139cb002e128f04fe0beb96d6cccba6e9e46ed7495270d28cbd Jul 15 10:21:03.179686 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.179666 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pdhc5"] Jul 15 10:21:03.182300 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:03.182280 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa664fba_4556_458c_9496_dc4cd0338f13.slice/crio-eb888eccd4b671d186f6b089ae06db3414f00fb3e2942b0dda75ff078b6f7a96 WatchSource:0}: Error finding container eb888eccd4b671d186f6b089ae06db3414f00fb3e2942b0dda75ff078b6f7a96: Status 404 returned error can't find the container with id eb888eccd4b671d186f6b089ae06db3414f00fb3e2942b0dda75ff078b6f7a96 Jul 15 10:21:03.309359 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.309336 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pdhc5" event={"ID":"aa664fba-4556-458c-9496-dc4cd0338f13","Type":"ContainerStarted","Data":"eb888eccd4b671d186f6b089ae06db3414f00fb3e2942b0dda75ff078b6f7a96"} Jul 15 10:21:03.310375 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.310348 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6z7nq" event={"ID":"c0b6c7bc-5908-4525-aacc-0b8c3186c646","Type":"ContainerStarted","Data":"11edc478434dab86cbe550b17f159fe2aacf9fe79ebe09b7c49135d82c080f9d"} Jul 15 10:21:03.311675 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.311655 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zgjkv" event={"ID":"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d","Type":"ContainerStarted","Data":"770a475832a738440040cef53e1878e05c62bbf3808ddfd20451938b11f78375"} Jul 15 10:21:03.311771 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.311689 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zgjkv" event={"ID":"ba5fe3cb-b4c9-44b1-a2bf-b1eb746eb81d","Type":"ContainerStarted","Data":"eb66049a4b710139cb002e128f04fe0beb96d6cccba6e9e46ed7495270d28cbd"} Jul 15 10:21:03.311771 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.311732 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:21:03.875462 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.875422 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:03.875635 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.875498 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:21:03.875635 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.875535 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:21:03.875635 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.875563 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:21:03.875635 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.875600 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:03.876564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.876527 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccad75-c598-469a-adb8-10b2cb51c8e9-service-ca-bundle\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:03.878423 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.878399 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"image-registry-5bf4bcd5c9-drqg5\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:21:03.878523 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.878403 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hszc4\" (UID: \"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:21:03.878523 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.878466 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b941075e-602c-47fe-b101-c38edae9c86c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7bgmk\" (UID: \"b941075e-602c-47fe-b101-c38edae9c86c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:21:03.878523 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.878476 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ccad75-c598-469a-adb8-10b2cb51c8e9-metrics-certs\") pod \"router-default-7d6976c5c8-mk6gv\" (UID: \"a2ccad75-c598-469a-adb8-10b2cb51c8e9\") " pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:03.976988 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.976950 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:21:03.979552 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:03.979531 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae01fde5-5868-4c83-9b4d-9b8e52cfd453-metrics-tls\") pod \"dns-default-kxkx5\" (UID: \"ae01fde5-5868-4c83-9b4d-9b8e52cfd453\") " pod="openshift-dns/dns-default-kxkx5" Jul 15 10:21:04.072829 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.072803 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qn4gb\"" Jul 15 10:21:04.079726 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.079704 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-slrfp\"" Jul 15 10:21:04.080861 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.080789 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:21:04.088646 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.088623 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" Jul 15 10:21:04.110882 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.110704 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-q7jrv\"" Jul 15 10:21:04.118918 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.118821 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:04.129630 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.129470 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xknbw\"" Jul 15 10:21:04.137652 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.137613 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" Jul 15 10:21:04.168183 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.167936 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jf9sn\"" Jul 15 10:21:04.176284 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.176176 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kxkx5" Jul 15 10:21:04.307555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.306011 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zgjkv" podStartSLOduration=67.305987697 podStartE2EDuration="1m7.305987697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:03.328109349 +0000 UTC m=+66.844101833" watchObservedRunningTime="2026-07-15 10:21:04.305987697 +0000 UTC m=+67.821980186" Jul 15 10:21:04.307674 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.307652 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bf4bcd5c9-drqg5"] Jul 15 10:21:04.310583 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:04.310550 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10a8b3c_1033_4a14_a886_7b3a1503a92f.slice/crio-0aefeb3c17f6bbf8e44185c9ceb234f72784b3035fa7bcd12435afa4586968d1 WatchSource:0}: Error finding container 0aefeb3c17f6bbf8e44185c9ceb234f72784b3035fa7bcd12435afa4586968d1: Status 404 returned error can't find the container with id 0aefeb3c17f6bbf8e44185c9ceb234f72784b3035fa7bcd12435afa4586968d1 Jul 15 10:21:04.328182 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.328099 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" event={"ID":"f10a8b3c-1033-4a14-a886-7b3a1503a92f","Type":"ContainerStarted","Data":"0aefeb3c17f6bbf8e44185c9ceb234f72784b3035fa7bcd12435afa4586968d1"} Jul 15 10:21:04.362823 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.362744 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4"] Jul 15 10:21:04.407602 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.407453 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d6976c5c8-mk6gv"] Jul 15 10:21:04.645575 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.645548 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kxkx5"] Jul 15 10:21:04.645648 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:04.645619 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk"] Jul 15 10:21:05.337033 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.336932 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pdhc5" event={"ID":"aa664fba-4556-458c-9496-dc4cd0338f13","Type":"ContainerStarted","Data":"4959e59b384439e97a89f784d6bcc8c9d7714770d3d02abab08162a767990fd8"} Jul 15 10:21:05.337033 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.336996 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pdhc5" event={"ID":"aa664fba-4556-458c-9496-dc4cd0338f13","Type":"ContainerStarted","Data":"4728ec1c223baeec52b0d5668c47b138e0ed1f37ef4c9106015715046a773ea2"} Jul 15 10:21:05.338731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.338669 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" event={"ID":"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581","Type":"ContainerStarted","Data":"9e1396ad80a9d5c067f6f6bf1f4c1e0ae4d0ef98c4b106728fe79ff69268082e"} Jul 15 10:21:05.341628 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.341157 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" event={"ID":"f10a8b3c-1033-4a14-a886-7b3a1503a92f","Type":"ContainerStarted","Data":"1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b"} Jul 15 10:21:05.341628 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.341561 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:21:05.342710 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.342686 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kxkx5" event={"ID":"ae01fde5-5868-4c83-9b4d-9b8e52cfd453","Type":"ContainerStarted","Data":"fee5c17ed1a400ea769cde3a3f99d447555b15ea333347ecfa2e35acf4dc06f9"} Jul 15 10:21:05.344148 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.344088 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" event={"ID":"b941075e-602c-47fe-b101-c38edae9c86c","Type":"ContainerStarted","Data":"738f3d9782d39685be90ee59187a2903fa66112ad8fad7bdd0c1d948b473de91"} Jul 15 10:21:05.346122 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.345646 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" event={"ID":"a2ccad75-c598-469a-adb8-10b2cb51c8e9","Type":"ContainerStarted","Data":"d5f714002a43b6d6ea9fdd3c0ebfbfaa7335a2c798b265b656b5520594c48bc8"} Jul 15 10:21:05.346122 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.345684 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" event={"ID":"a2ccad75-c598-469a-adb8-10b2cb51c8e9","Type":"ContainerStarted","Data":"1df360461488a29fac25b05d38e57c72ba0ea9eb60abb80c2f830844cf2dd052"} Jul 15 10:21:05.366537 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.366479 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pdhc5" podStartSLOduration=67.757235505 podStartE2EDuration="1m9.366464019s" totalImagesPullingTime="1.609228514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:05.365324639 +0000 UTC m=+68.881317136" watchObservedRunningTime="2026-07-15 10:21:05.366464019 +0000 UTC m=+68.882456504" Jul 15 10:21:05.388093 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.388026 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" podStartSLOduration=61.388010489 podStartE2EDuration="1m1.388010489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:05.38653989 +0000 UTC m=+68.902532376" watchObservedRunningTime="2026-07-15 10:21:05.388010489 +0000 UTC m=+68.904002972" Jul 15 10:21:05.408479 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:05.408413 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" podStartSLOduration=68.40839683 podStartE2EDuration="1m8.40839683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:05.408296134 +0000 UTC m=+68.924288620" watchObservedRunningTime="2026-07-15 10:21:05.40839683 +0000 UTC m=+68.924389315" Jul 15 10:21:06.120143 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:06.120103 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:06.123315 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:06.123121 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:06.353050 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:06.352997 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:06.354730 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:06.354705 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d6976c5c8-mk6gv" Jul 15 10:21:11.374746 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.374711 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kxkx5" event={"ID":"ae01fde5-5868-4c83-9b4d-9b8e52cfd453","Type":"ContainerStarted","Data":"fd0918f67e5b052a7c3c5da46749b62bc6b68dd2c926081589c3a05357af91dc"} Jul 15 10:21:11.376594 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.376548 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kxkx5" event={"ID":"ae01fde5-5868-4c83-9b4d-9b8e52cfd453","Type":"ContainerStarted","Data":"5fc52cda142c6b86f745a7f7a80b81c7eb6411886e5c6a7c8809d74f06acd98f"} Jul 15 10:21:11.376594 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.376591 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kxkx5" Jul 15 10:21:11.378207 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.378181 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" event={"ID":"b941075e-602c-47fe-b101-c38edae9c86c","Type":"ContainerStarted","Data":"a84afe33335f739647c1a6afb887ee79c9178cb6669e4d848b3930db246e396e"} Jul 15 10:21:11.378289 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.378238 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" event={"ID":"b941075e-602c-47fe-b101-c38edae9c86c","Type":"ContainerStarted","Data":"af7ea7dd5f411f07ce6f4c5f1d6d743cea8f0fcf1a0a5677aaef8de336ceebb6"} Jul 15 10:21:11.379712 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.379688 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6z7nq" event={"ID":"c0b6c7bc-5908-4525-aacc-0b8c3186c646","Type":"ContainerStarted","Data":"4d561a15ccfa23370603ae5e9de23766585f0d3f07b3333110a33c08ed176df2"} Jul 15 10:21:11.381579 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.381552 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" event={"ID":"b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581","Type":"ContainerStarted","Data":"d5c8dbe896c3442d8275aeaf68abfb440808695cd25f40281bda9dcc89ceb6f5"} Jul 15 10:21:11.383564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.383544 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hzjrv" event={"ID":"37035e8a-7489-4a0a-97fc-dbb3739cc852","Type":"ContainerStarted","Data":"21408084c69327fd5816865785f0713b524287d36b2b73bb100491bf3647d5cf"} Jul 15 10:21:11.399411 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.399321 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kxkx5" podStartSLOduration=34.189311916 podStartE2EDuration="40.399307275s" totalImagesPullingTime="6.209995359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:11.397902886 +0000 UTC m=+74.913895370" watchObservedRunningTime="2026-07-15 10:21:11.399307275 +0000 UTC m=+74.915299760" Jul 15 10:21:11.415313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.415275 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7bgmk" podStartSLOduration=58.236249148 podStartE2EDuration="1m4.415262663s" totalImagesPullingTime="6.179013515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:11.4140749 +0000 UTC m=+74.930067386" watchObservedRunningTime="2026-07-15 10:21:11.415262663 +0000 UTC m=+74.931255196" Jul 15 10:21:11.458220 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.458151 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6z7nq" podStartSLOduration=66.586326429 podStartE2EDuration="1m14.458138317s" totalImagesPullingTime="7.871811888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:11.45733322 +0000 UTC m=+74.973325705" watchObservedRunningTime="2026-07-15 10:21:11.458138317 +0000 UTC m=+74.974130826" Jul 15 10:21:11.458561 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:11.458541 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hszc4" podStartSLOduration=61.25169178 podStartE2EDuration="1m7.458536544s" totalImagesPullingTime="6.206844764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:11.434369301 +0000 UTC m=+74.950361789" watchObservedRunningTime="2026-07-15 10:21:11.458536544 +0000 UTC m=+74.974529028" Jul 15 10:21:12.557685 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.557653 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-679b859b49-4hkkq"] Jul 15 10:21:12.574705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.574658 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679b859b49-4hkkq"] Jul 15 10:21:12.574871 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.574756 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.577566 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.577388 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 10:21:12.577566 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.577433 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 10:21:12.577566 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.577500 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 10:21:12.577810 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.577620 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 10:21:12.579048 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.578640 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 10:21:12.579048 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.578665 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 10:21:12.579048 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.578699 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 10:21:12.579048 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.578931 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6zk45\"" Jul 15 10:21:12.660567 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.660531 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-oauth-serving-cert\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.660702 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.660581 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9bwj\" (UniqueName: \"kubernetes.io/projected/e5c8ea1b-25f1-4d12-981c-458129fbc84f-kube-api-access-m9bwj\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.660702 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.660610 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-service-ca\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.660702 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.660655 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-serving-cert\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.660702 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.660679 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-config\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.660871 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.660733 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-oauth-config\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.761846 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.761808 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-oauth-serving-cert\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.762045 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.761861 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9bwj\" (UniqueName: \"kubernetes.io/projected/e5c8ea1b-25f1-4d12-981c-458129fbc84f-kube-api-access-m9bwj\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.762045 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.761889 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-service-ca\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.762045 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.761926 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-serving-cert\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.762045 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.761945 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-config\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.762289 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.762143 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-oauth-config\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.762649 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.762619 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-oauth-serving-cert\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.762649 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.762635 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-service-ca\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.762814 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.762666 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-config\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.764610 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.764588 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-serving-cert\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.764750 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.764731 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-oauth-config\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.772313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.772290 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9bwj\" (UniqueName: \"kubernetes.io/projected/e5c8ea1b-25f1-4d12-981c-458129fbc84f-kube-api-access-m9bwj\") pod \"console-679b859b49-4hkkq\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:12.887190 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:12.887114 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:13.056054 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:13.055894 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679b859b49-4hkkq"] Jul 15 10:21:13.060873 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:13.059096 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c8ea1b_25f1_4d12_981c_458129fbc84f.slice/crio-a198cead4012637bac9f26e35696ee01c6c8ad1b4201c40554d1998d98b69199 WatchSource:0}: Error finding container a198cead4012637bac9f26e35696ee01c6c8ad1b4201c40554d1998d98b69199: Status 404 returned error can't find the container with id a198cead4012637bac9f26e35696ee01c6c8ad1b4201c40554d1998d98b69199 Jul 15 10:21:13.398226 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:13.398190 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679b859b49-4hkkq" event={"ID":"e5c8ea1b-25f1-4d12-981c-458129fbc84f","Type":"ContainerStarted","Data":"a198cead4012637bac9f26e35696ee01c6c8ad1b4201c40554d1998d98b69199"} Jul 15 10:21:16.412234 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:16.412193 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hzjrv" event={"ID":"37035e8a-7489-4a0a-97fc-dbb3739cc852","Type":"ContainerStarted","Data":"d9c4d7aa14decf873f63e20f690ccf9776d36dc57bb2b2805ebc11d50c954f93"} Jul 15 10:21:16.434219 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:16.434169 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hzjrv" podStartSLOduration=18.342741313 podStartE2EDuration="36.434154978s" totalImagesPullingTime="18.091413665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:20:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:16.431982889 +0000 UTC m=+79.947975376" watchObservedRunningTime="2026-07-15 10:21:16.434154978 +0000 UTC m=+79.950147517" Jul 15 10:21:18.423439 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:18.423406 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679b859b49-4hkkq" event={"ID":"e5c8ea1b-25f1-4d12-981c-458129fbc84f","Type":"ContainerStarted","Data":"a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df"} Jul 15 10:21:18.447134 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:18.447082 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-679b859b49-4hkkq" podStartSLOduration=1.733792657 podStartE2EDuration="6.44704553s" totalImagesPullingTime="4.713252873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:18.446641114 +0000 UTC m=+81.962633613" watchObservedRunningTime="2026-07-15 10:21:18.44704553 +0000 UTC m=+81.963038055" Jul 15 10:21:19.722439 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.722335 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rk8np"] Jul 15 10:21:19.724955 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.724935 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.727457 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.727434 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 15 10:21:19.728277 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.728258 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 15 10:21:19.728373 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.728341 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 10:21:19.728425 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.728414 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bq54f\"" Jul 15 10:21:19.735404 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.735380 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rk8np"] Jul 15 10:21:19.813666 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.813637 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwwnz\" (UniqueName: \"kubernetes.io/projected/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-kube-api-access-jwwnz\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.813776 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.813675 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.813776 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.813709 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.813776 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.813765 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.914579 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.914556 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwwnz\" (UniqueName: \"kubernetes.io/projected/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-kube-api-access-jwwnz\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.914674 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.914585 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.914674 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.914606 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.914674 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.914636 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.915294 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.915274 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.916822 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.916801 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.916893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.916879 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:19.925129 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:19.925111 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwwnz\" (UniqueName: \"kubernetes.io/projected/69f3869f-2528-4450-96c9-f3d4f3b2ae2c-kube-api-access-jwwnz\") pod \"prometheus-operator-64785b6675-rk8np\" (UID: \"69f3869f-2528-4450-96c9-f3d4f3b2ae2c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:20.033433 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:20.033403 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" Jul 15 10:21:20.159146 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:20.159116 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rk8np"] Jul 15 10:21:20.162162 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:20.162131 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69f3869f_2528_4450_96c9_f3d4f3b2ae2c.slice/crio-1d7ac46497485aabaa8cff280890d9894238e69f2444d9c97aa02a2a72e805e1 WatchSource:0}: Error finding container 1d7ac46497485aabaa8cff280890d9894238e69f2444d9c97aa02a2a72e805e1: Status 404 returned error can't find the container with id 1d7ac46497485aabaa8cff280890d9894238e69f2444d9c97aa02a2a72e805e1 Jul 15 10:21:20.432437 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:20.432380 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" event={"ID":"69f3869f-2528-4450-96c9-f3d4f3b2ae2c","Type":"ContainerStarted","Data":"1d7ac46497485aabaa8cff280890d9894238e69f2444d9c97aa02a2a72e805e1"} Jul 15 10:21:21.395594 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:21.395565 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kxkx5" Jul 15 10:21:22.443135 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:22.443093 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" event={"ID":"69f3869f-2528-4450-96c9-f3d4f3b2ae2c","Type":"ContainerStarted","Data":"4c9050b905cdb3da08d53acfe0e221f7c379b401ccf206465e4764bce98ed75d"} Jul 15 10:21:22.443626 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:22.443169 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" event={"ID":"69f3869f-2528-4450-96c9-f3d4f3b2ae2c","Type":"ContainerStarted","Data":"335dd5e19dcb722fbc9cdb62b32fff010fc2569b86050a8f48ebead3f828129d"} Jul 15 10:21:22.464775 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:22.464727 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-rk8np" podStartSLOduration=1.9885065690000001 podStartE2EDuration="3.464709872s" totalImagesPullingTime="1.476203303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:22.462377381 +0000 UTC m=+85.978369867" watchObservedRunningTime="2026-07-15 10:21:22.464709872 +0000 UTC m=+85.980702356" Jul 15 10:21:22.888214 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:22.888179 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:22.888341 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:22.888241 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:22.893003 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:22.892979 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:23.453795 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:23.453750 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:24.144411 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.144383 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gfkx2"] Jul 15 10:21:24.147970 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.147950 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.150477 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.150456 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 10:21:24.150599 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.150503 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w2c94\"" Jul 15 10:21:24.150599 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.150517 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 10:21:24.151045 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.151025 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 10:21:24.244622 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244598 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-metrics-client-ca\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.244748 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244634 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hnbw\" (UniqueName: \"kubernetes.io/projected/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-kube-api-access-4hnbw\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.244748 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244668 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-sys\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.244748 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244710 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-tls\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.244748 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244742 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.245174 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244789 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-root\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.245174 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244814 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-textfile\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.245174 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244842 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.245174 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.244907 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-wtmp\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.345718 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345688 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-root\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.345827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345723 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-textfile\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.345827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345747 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.345827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345773 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-wtmp\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.345827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345808 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-root\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345811 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-metrics-client-ca\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345886 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hnbw\" (UniqueName: \"kubernetes.io/projected/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-kube-api-access-4hnbw\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345953 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-sys\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345972 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-wtmp\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.345978 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-tls\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346269 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.346021 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346269 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.346083 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-sys\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346269 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.346101 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-textfile\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346423 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.346396 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.346474 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.346430 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-metrics-client-ca\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.348100 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.348081 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-tls\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.348188 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.348162 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.355817 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.355796 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hnbw\" (UniqueName: \"kubernetes.io/projected/1e32521d-cbf5-49f0-b1f3-7e74d215f5c0-kube-api-access-4hnbw\") pod \"node-exporter-gfkx2\" (UID: \"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0\") " pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.457409 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:24.457351 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gfkx2" Jul 15 10:21:24.468578 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:24.468556 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e32521d_cbf5_49f0_b1f3_7e74d215f5c0.slice/crio-5e215fb7f0aa8b0c57c0db22d57cf5367f31b9b92cd2132c6c338cda1e3e8385 WatchSource:0}: Error finding container 5e215fb7f0aa8b0c57c0db22d57cf5367f31b9b92cd2132c6c338cda1e3e8385: Status 404 returned error can't find the container with id 5e215fb7f0aa8b0c57c0db22d57cf5367f31b9b92cd2132c6c338cda1e3e8385 Jul 15 10:21:25.207142 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.207108 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 10:21:25.212599 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.212569 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.215404 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215382 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 10:21:25.215686 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215664 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 10:21:25.215795 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215382 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 10:21:25.215795 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215470 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 10:21:25.215864 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215454 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 10:21:25.215898 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215489 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 10:21:25.215943 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215493 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 10:21:25.216007 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215495 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 10:21:25.216082 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215522 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nkqnm\"" Jul 15 10:21:25.216156 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.215538 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 10:21:25.224459 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.224435 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 10:21:25.353339 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353306 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-out\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353356 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353381 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl8bs\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-kube-api-access-wl8bs\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353404 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353419 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-web-config\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353438 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353459 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-volume\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353475 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353762 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353497 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353762 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353555 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353762 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353616 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353762 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353660 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.353762 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.353697 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454017 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.453995 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454017 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454020 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454043 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-out\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454087 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454116 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl8bs\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-kube-api-access-wl8bs\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454142 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-web-config\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454183 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454207 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-volume\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454528 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454232 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454528 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454258 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454528 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454283 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454528 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454305 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.454703 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.454552 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.455324 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:21:25.454950 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle podName:2f17b7dd-03fe-4e31-818e-6513b4e5baac nodeName:}" failed. No retries permitted until 2026-07-15 10:21:25.954929075 +0000 UTC m=+89.470921547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac") : configmap references non-existent config key: ca-bundle.crt Jul 15 10:21:25.455324 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:21:25.455286 2585 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 15 10:21:25.455527 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:21:25.455339 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls podName:2f17b7dd-03fe-4e31-818e-6513b4e5baac nodeName:}" failed. No retries permitted until 2026-07-15 10:21:25.95532175 +0000 UTC m=+89.471314215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac") : secret "alertmanager-main-tls" not found Jul 15 10:21:25.457408 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.457289 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gfkx2" event={"ID":"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0","Type":"ContainerStarted","Data":"5e215fb7f0aa8b0c57c0db22d57cf5367f31b9b92cd2132c6c338cda1e3e8385"} Jul 15 10:21:25.457408 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.457317 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.457408 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.457316 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-out\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.457892 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.457630 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.458423 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.458382 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.458520 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.458482 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.458739 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.458718 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.458817 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.458779 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-web-config\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.458956 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.458933 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-volume\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.459081 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.459042 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.470798 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.470776 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl8bs\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-kube-api-access-wl8bs\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.958466 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.958433 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.958634 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.958501 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.959287 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.959262 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:25.960925 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:25.960892 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:26.124257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:26.124230 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:21:26.294045 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:26.294021 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 10:21:26.295327 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:26.295295 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f17b7dd_03fe_4e31_818e_6513b4e5baac.slice/crio-a6e78dbe8d6ebc234d5307c1eafb808407602d6f2f6b1dff9dbb146319002f83 WatchSource:0}: Error finding container a6e78dbe8d6ebc234d5307c1eafb808407602d6f2f6b1dff9dbb146319002f83: Status 404 returned error can't find the container with id a6e78dbe8d6ebc234d5307c1eafb808407602d6f2f6b1dff9dbb146319002f83 Jul 15 10:21:26.357045 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:26.357024 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:21:26.462749 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:26.462719 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerStarted","Data":"a6e78dbe8d6ebc234d5307c1eafb808407602d6f2f6b1dff9dbb146319002f83"} Jul 15 10:21:27.467299 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:27.467272 2585 generic.go:356] "Generic (PLEG): container finished" podID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerID="a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b" exitCode=0 Jul 15 10:21:27.467659 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:27.467337 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerDied","Data":"a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b"} Jul 15 10:21:28.476177 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:28.476144 2585 generic.go:356] "Generic (PLEG): container finished" podID="1e32521d-cbf5-49f0-b1f3-7e74d215f5c0" containerID="3ea0a387ae7ed9b9e11dc0dce8c64266659e2b01923328d59fe33d0a53f5c791" exitCode=0 Jul 15 10:21:28.476566 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:28.476223 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gfkx2" event={"ID":"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0","Type":"ContainerDied","Data":"3ea0a387ae7ed9b9e11dc0dce8c64266659e2b01923328d59fe33d0a53f5c791"} Jul 15 10:21:28.815104 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:28.815006 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679b859b49-4hkkq"] Jul 15 10:21:29.394998 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.394919 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq"] Jul 15 10:21:29.398427 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.398407 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.400660 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.400632 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 10:21:29.400832 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.400811 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 10:21:29.400993 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.400978 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 10:21:29.401089 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.401052 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 10:21:29.401146 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.401052 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 10:21:29.401207 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.401158 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-h4cxn\"" Jul 15 10:21:29.407859 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.407831 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 10:21:29.411421 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.411400 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq"] Jul 15 10:21:29.482244 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.482212 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gfkx2" event={"ID":"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0","Type":"ContainerStarted","Data":"8f4986ffcf4f2c1fb629c881f7f621cda7cae44683f9f0b059c43c40d955806b"} Jul 15 10:21:29.482555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.482251 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gfkx2" event={"ID":"1e32521d-cbf5-49f0-b1f3-7e74d215f5c0","Type":"ContainerStarted","Data":"c2ac2f0b91357dff29c72a9578da90db20dec3ad25088b400f8db59089d1dc94"} Jul 15 10:21:29.485203 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.485186 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerStarted","Data":"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383"} Jul 15 10:21:29.485268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.485227 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerStarted","Data":"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6"} Jul 15 10:21:29.485268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.485257 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerStarted","Data":"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3"} Jul 15 10:21:29.485341 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.485281 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerStarted","Data":"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e"} Jul 15 10:21:29.485341 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.485304 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerStarted","Data":"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2"} Jul 15 10:21:29.485908 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.485891 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-federate-client-tls\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.485944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.485926 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-metrics-client-ca\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.485984 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.485965 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj8fm\" (UniqueName: \"kubernetes.io/projected/3ac9e27e-5c04-4865-8a90-d6519decb39e-kube-api-access-wj8fm\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.486151 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.486128 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-serving-certs-ca-bundle\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.486197 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.486165 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-telemeter-client-tls\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.486263 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.486250 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.486310 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.486293 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.486358 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.486311 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-secret-telemeter-client\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.506472 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.506432 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gfkx2" podStartSLOduration=1.658994069 podStartE2EDuration="5.5064208s" totalImagesPullingTime="3.784003591s" totalInitContainerRuntime="63.42314ms" isStatefulPod=true podCreationTimestamp="2026-07-15 10:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:29.505783453 +0000 UTC m=+93.021775938" watchObservedRunningTime="2026-07-15 10:21:29.5064208 +0000 UTC m=+93.022413285" Jul 15 10:21:29.586864 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.586842 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-federate-client-tls\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.586963 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.586872 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-metrics-client-ca\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.586963 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.586889 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8fm\" (UniqueName: \"kubernetes.io/projected/3ac9e27e-5c04-4865-8a90-d6519decb39e-kube-api-access-wj8fm\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.586963 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.586930 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-serving-certs-ca-bundle\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.586963 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.586953 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-telemeter-client-tls\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.587184 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.586996 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.587184 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.587034 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.587184 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.587168 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-secret-telemeter-client\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.587652 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.587627 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-metrics-client-ca\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.587957 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.587937 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-serving-certs-ca-bundle\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.588116 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.588096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac9e27e-5c04-4865-8a90-d6519decb39e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.589567 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.589544 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-telemeter-client-tls\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.589659 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.589636 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-secret-telemeter-client\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.589849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.589833 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.589938 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.589919 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ac9e27e-5c04-4865-8a90-d6519decb39e-federate-client-tls\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.600657 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.600635 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj8fm\" (UniqueName: \"kubernetes.io/projected/3ac9e27e-5c04-4865-8a90-d6519decb39e-kube-api-access-wj8fm\") pod \"telemeter-client-5bd7dd4f65-x77kq\" (UID: \"3ac9e27e-5c04-4865-8a90-d6519decb39e\") " pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.712541 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.712486 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" Jul 15 10:21:29.860001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.859971 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq"] Jul 15 10:21:29.940407 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:29.940372 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac9e27e_5c04_4865_8a90_d6519decb39e.slice/crio-639ace8725076439e61a63af6ce85e709c14fa07f763d3c094e4e6a7fc188cfb WatchSource:0}: Error finding container 639ace8725076439e61a63af6ce85e709c14fa07f763d3c094e4e6a7fc188cfb: Status 404 returned error can't find the container with id 639ace8725076439e61a63af6ce85e709c14fa07f763d3c094e4e6a7fc188cfb Jul 15 10:21:29.958244 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.958215 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-675b5db768-wb7m7"] Jul 15 10:21:29.961346 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.961326 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:29.967944 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.967885 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 10:21:29.972368 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.972349 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-675b5db768-wb7m7"] Jul 15 10:21:29.989954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.989936 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-service-ca\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:29.990028 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.989965 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-serving-cert\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:29.990028 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.989980 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-config\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:29.990114 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.990044 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-oauth-serving-cert\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:29.990114 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.990089 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvfbh\" (UniqueName: \"kubernetes.io/projected/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-kube-api-access-xvfbh\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:29.990175 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.990116 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-trusted-ca-bundle\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:29.990175 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:29.990158 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-oauth-config\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.090910 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.090885 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-oauth-serving-cert\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.090994 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.090918 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvfbh\" (UniqueName: \"kubernetes.io/projected/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-kube-api-access-xvfbh\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.090994 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.090948 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-trusted-ca-bundle\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.090994 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.090976 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-oauth-config\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.091165 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.090999 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-service-ca\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.091165 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.091019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-serving-cert\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.091165 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.091048 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-config\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.091749 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.091722 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-service-ca\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.091877 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.091856 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-oauth-serving-cert\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.091935 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.091854 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-config\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.092095 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.092048 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-trusted-ca-bundle\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.093402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.093350 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-oauth-config\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.093541 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.093513 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-serving-cert\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.099782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.099760 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvfbh\" (UniqueName: \"kubernetes.io/projected/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-kube-api-access-xvfbh\") pod \"console-675b5db768-wb7m7\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.270791 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.270759 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:30.377338 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.377297 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:21:30.383405 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.383378 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.386044 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.386019 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.386494 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.386639 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.386689 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.386736 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.386960 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387017 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387055 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387316 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387378 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6rfbq4tghgn9b\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387447 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-dwfcv\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387488 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387741 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387846 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.387869 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 15 10:21:30.389769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.388100 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 15 10:21:30.390626 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.390607 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395176 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395224 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395250 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djw8g\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-kube-api-access-djw8g\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395287 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395329 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-config\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395350 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395374 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395401 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395429 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395472 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395526 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395550 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395580 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395633 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395697 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395719 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396920 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395742 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396920 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395780 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396920 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395822 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.396920 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.395843 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.397128 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.397110 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:21:30.425087 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.425043 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-675b5db768-wb7m7"] Jul 15 10:21:30.492862 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.492834 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerStarted","Data":"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c"} Jul 15 10:21:30.494338 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.494318 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675b5db768-wb7m7" event={"ID":"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f","Type":"ContainerStarted","Data":"74baf3bae3acd597419fde0ab97baa1afdbbfab2b11203e5da6352f971e4765a"} Jul 15 10:21:30.495583 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.495557 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" event={"ID":"3ac9e27e-5c04-4865-8a90-d6519decb39e","Type":"ContainerStarted","Data":"639ace8725076439e61a63af6ce85e709c14fa07f763d3c094e4e6a7fc188cfb"} Jul 15 10:21:30.496643 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.496619 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-config\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.496724 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.496660 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.496724 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.496684 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.496724 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.496700 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.496724 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.496720 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.496897 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.496779 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497176 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497139 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497262 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497192 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497262 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497225 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497369 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497282 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497418 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497380 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497467 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497419 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497467 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497565 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497496 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497565 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497535 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497666 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497605 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497666 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497646 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497753 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497692 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497753 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497720 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497753 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497746 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djw8g\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-kube-api-access-djw8g\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.497899 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.497825 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.498756 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.498729 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.498932 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.498909 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.500610 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.500349 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.500937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.500805 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.501203 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.501179 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.501375 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.501333 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.501451 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.501379 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-config\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.501789 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.501534 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.501789 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.501741 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.502363 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.502340 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.502736 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.502713 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.502841 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.502767 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.502910 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.502842 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.502965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.502927 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.503020 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.502958 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.503780 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.503761 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.504020 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.504003 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.504307 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.504284 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.508884 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.508864 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djw8g\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-kube-api-access-djw8g\") pod \"prometheus-k8s-0\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.528864 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.528774 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.239662461 podStartE2EDuration="5.528756428s" totalImagesPullingTime="3.26999754s" totalInitContainerRuntime="19.096427ms" isStatefulPod=false podCreationTimestamp="2026-07-15 10:21:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:30.528400722 +0000 UTC m=+94.044393210" watchObservedRunningTime="2026-07-15 10:21:30.528756428 +0000 UTC m=+94.044748915" Jul 15 10:21:30.698402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.698369 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:30.862338 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:30.862305 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:21:30.865928 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:21:30.865891 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea1fa78_536e_456d_9083_cc9019fe40b8.slice/crio-316668b481a83557b198250592cb5d3ec6b54a80e1f6734f19d6a731ee09fd4a WatchSource:0}: Error finding container 316668b481a83557b198250592cb5d3ec6b54a80e1f6734f19d6a731ee09fd4a: Status 404 returned error can't find the container with id 316668b481a83557b198250592cb5d3ec6b54a80e1f6734f19d6a731ee09fd4a Jul 15 10:21:31.501445 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:31.501406 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675b5db768-wb7m7" event={"ID":"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f","Type":"ContainerStarted","Data":"c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b"} Jul 15 10:21:31.502950 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:31.502927 2585 generic.go:356] "Generic (PLEG): container finished" podID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" exitCode=0 Jul 15 10:21:31.503039 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:31.503010 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerDied","Data":"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d"} Jul 15 10:21:31.503098 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:31.503087 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerStarted","Data":"316668b481a83557b198250592cb5d3ec6b54a80e1f6734f19d6a731ee09fd4a"} Jul 15 10:21:31.521272 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:31.521232 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-675b5db768-wb7m7" podStartSLOduration=2.521220261 podStartE2EDuration="2.521220261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:31.520786736 +0000 UTC m=+95.036779232" watchObservedRunningTime="2026-07-15 10:21:31.521220261 +0000 UTC m=+95.037212743" Jul 15 10:21:32.126107 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:32.126052 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bf4bcd5c9-drqg5"] Jul 15 10:21:32.509801 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:32.509768 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" event={"ID":"3ac9e27e-5c04-4865-8a90-d6519decb39e","Type":"ContainerStarted","Data":"0d4a6da8301757d7f2871e4c5d03b46497151e349fc33981fd833a04437d4e76"} Jul 15 10:21:33.517262 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:33.517222 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" event={"ID":"3ac9e27e-5c04-4865-8a90-d6519decb39e","Type":"ContainerStarted","Data":"b639009e69eba139ca99bb167f820eca5f088a40bef6a3308059e3f164e141c6"} Jul 15 10:21:33.517634 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:33.517291 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" event={"ID":"3ac9e27e-5c04-4865-8a90-d6519decb39e","Type":"ContainerStarted","Data":"74fd5f5ce425febe38d45d4a36d8bce322d87729b993803e0a52caf3d27af860"} Jul 15 10:21:33.544934 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:33.544873 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5bd7dd4f65-x77kq" podStartSLOduration=2.106294891 podStartE2EDuration="4.544850823s" totalImagesPullingTime="2.438555932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:33.542200633 +0000 UTC m=+97.058193117" watchObservedRunningTime="2026-07-15 10:21:33.544850823 +0000 UTC m=+97.060843309" Jul 15 10:21:34.294581 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:34.293100 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-675b5db768-wb7m7"] Jul 15 10:21:34.330824 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:34.330791 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zgjkv" Jul 15 10:21:35.531177 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:35.531087 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerStarted","Data":"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4"} Jul 15 10:21:35.531177 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:35.531163 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerStarted","Data":"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e"} Jul 15 10:21:38.549291 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:38.549257 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerStarted","Data":"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079"} Jul 15 10:21:38.549627 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:38.549321 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerStarted","Data":"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db"} Jul 15 10:21:38.549627 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:38.549349 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerStarted","Data":"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0"} Jul 15 10:21:38.549627 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:38.549376 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerStarted","Data":"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a"} Jul 15 10:21:38.585099 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:38.585030 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.5609957899999998 podStartE2EDuration="8.585015361s" totalImagesPullingTime="6.005168647s" totalInitContainerRuntime="18.850924ms" isStatefulPod=false podCreationTimestamp="2026-07-15 10:21:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:21:38.582781202 +0000 UTC m=+102.098773688" watchObservedRunningTime="2026-07-15 10:21:38.585015361 +0000 UTC m=+102.101007848" Jul 15 10:21:40.271746 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:40.271713 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:40.698825 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:40.698751 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:21:51.607272 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:51.607233 2585 generic.go:356] "Generic (PLEG): container finished" podID="8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6" containerID="ad75f1ea820a66bc6744fad68756c0a3e94cee900bbc39c6c71e78c727c7f0ff" exitCode=0 Jul 15 10:21:51.607674 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:51.607307 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" event={"ID":"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6","Type":"ContainerDied","Data":"ad75f1ea820a66bc6744fad68756c0a3e94cee900bbc39c6c71e78c727c7f0ff"} Jul 15 10:21:51.607674 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:51.607635 2585 scope.go:117] "RemoveContainer" containerID="ad75f1ea820a66bc6744fad68756c0a3e94cee900bbc39c6c71e78c727c7f0ff" Jul 15 10:21:52.613849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:52.613817 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-2m7mm" event={"ID":"8eac6d3c-3a8b-4fc0-92ef-b6ca6def64a6","Type":"ContainerStarted","Data":"8d95c4a9ee54c90fe671734705fbd106c7877eedd9cb391846e7c076576b9995"} Jul 15 10:21:53.842317 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:53.842271 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-679b859b49-4hkkq" podUID="e5c8ea1b-25f1-4d12-981c-458129fbc84f" containerName="console" containerID="cri-o://a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df" gracePeriod=15 Jul 15 10:21:54.094049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.094002 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:54.196125 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196101 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-config\") pod \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " Jul 15 10:21:54.196268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196157 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9bwj\" (UniqueName: \"kubernetes.io/projected/e5c8ea1b-25f1-4d12-981c-458129fbc84f-kube-api-access-m9bwj\") pod \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " Jul 15 10:21:54.196268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196175 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-service-ca\") pod \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " Jul 15 10:21:54.196268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196194 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-oauth-serving-cert\") pod \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " Jul 15 10:21:54.196268 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196227 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-serving-cert\") pod \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " Jul 15 10:21:54.196479 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196272 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-oauth-config\") pod \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\" (UID: \"e5c8ea1b-25f1-4d12-981c-458129fbc84f\") " Jul 15 10:21:54.196562 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196534 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-config" (OuterVolumeSpecName: "console-config") pod "e5c8ea1b-25f1-4d12-981c-458129fbc84f" (UID: "e5c8ea1b-25f1-4d12-981c-458129fbc84f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:54.196627 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196604 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-service-ca" (OuterVolumeSpecName: "service-ca") pod "e5c8ea1b-25f1-4d12-981c-458129fbc84f" (UID: "e5c8ea1b-25f1-4d12-981c-458129fbc84f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:54.196671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.196600 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e5c8ea1b-25f1-4d12-981c-458129fbc84f" (UID: "e5c8ea1b-25f1-4d12-981c-458129fbc84f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:54.198303 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.198267 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e5c8ea1b-25f1-4d12-981c-458129fbc84f" (UID: "e5c8ea1b-25f1-4d12-981c-458129fbc84f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:21:54.198397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.198315 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e5c8ea1b-25f1-4d12-981c-458129fbc84f" (UID: "e5c8ea1b-25f1-4d12-981c-458129fbc84f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:21:54.198397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.198382 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c8ea1b-25f1-4d12-981c-458129fbc84f-kube-api-access-m9bwj" (OuterVolumeSpecName: "kube-api-access-m9bwj") pod "e5c8ea1b-25f1-4d12-981c-458129fbc84f" (UID: "e5c8ea1b-25f1-4d12-981c-458129fbc84f"). InnerVolumeSpecName "kube-api-access-m9bwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:21:54.297013 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.296989 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-oauth-serving-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:54.297013 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.297009 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-serving-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:54.297164 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.297018 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-oauth-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:54.297164 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.297029 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-console-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:54.297164 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.297038 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9bwj\" (UniqueName: \"kubernetes.io/projected/e5c8ea1b-25f1-4d12-981c-458129fbc84f-kube-api-access-m9bwj\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:54.297164 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.297048 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c8ea1b-25f1-4d12-981c-458129fbc84f-service-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:54.623776 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.623747 2585 generic.go:356] "Generic (PLEG): container finished" podID="e5c8ea1b-25f1-4d12-981c-458129fbc84f" containerID="a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df" exitCode=0 Jul 15 10:21:54.623909 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.623782 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679b859b49-4hkkq" event={"ID":"e5c8ea1b-25f1-4d12-981c-458129fbc84f","Type":"ContainerDied","Data":"a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df"} Jul 15 10:21:54.623909 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.623798 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679b859b49-4hkkq" Jul 15 10:21:54.623909 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.623824 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679b859b49-4hkkq" event={"ID":"e5c8ea1b-25f1-4d12-981c-458129fbc84f","Type":"ContainerDied","Data":"a198cead4012637bac9f26e35696ee01c6c8ad1b4201c40554d1998d98b69199"} Jul 15 10:21:54.623909 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.623845 2585 scope.go:117] "RemoveContainer" containerID="a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df" Jul 15 10:21:54.633554 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.633534 2585 scope.go:117] "RemoveContainer" containerID="a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df" Jul 15 10:21:54.633826 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:21:54.633802 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df\": container with ID starting with a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df not found: ID does not exist" containerID="a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df" Jul 15 10:21:54.633883 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.633832 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df"} err="failed to get container status \"a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df\": rpc error: code = NotFound desc = could not find container \"a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df\": container with ID starting with a11ee95ae36d233160938d3a23f6d7e01ad3cb53189a935b5be04ba5b659d6df not found: ID does not exist" Jul 15 10:21:54.646946 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.646924 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679b859b49-4hkkq"] Jul 15 10:21:54.652966 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.652945 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-679b859b49-4hkkq"] Jul 15 10:21:54.932383 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:54.932321 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5c8ea1b-25f1-4d12-981c-458129fbc84f" path="/var/lib/kubelet/pods/e5c8ea1b-25f1-4d12-981c-458129fbc84f/volumes" Jul 15 10:21:57.147296 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.147206 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" podUID="f10a8b3c-1033-4a14-a886-7b3a1503a92f" containerName="registry" containerID="cri-o://1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b" gracePeriod=30 Jul 15 10:21:57.167547 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:21:57.167524 2585 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10a8b3c_1033_4a14_a886_7b3a1503a92f.slice/crio-1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b.scope\": RecentStats: unable to find data in memory cache]" Jul 15 10:21:57.384777 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.384754 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:21:57.524497 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.524474 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-certificates\") pod \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " Jul 15 10:21:57.524633 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.524509 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-trusted-ca\") pod \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " Jul 15 10:21:57.524633 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.524530 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") pod \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " Jul 15 10:21:57.524633 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.524547 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-image-registry-private-configuration\") pod \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " Jul 15 10:21:57.524633 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.524576 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-bound-sa-token\") pod \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " Jul 15 10:21:57.524633 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.524601 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f10a8b3c-1033-4a14-a886-7b3a1503a92f-ca-trust-extracted\") pod \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " Jul 15 10:21:57.524633 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.524618 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg5gm\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-kube-api-access-zg5gm\") pod \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " Jul 15 10:21:57.524943 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.524652 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-installation-pull-secrets\") pod \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\" (UID: \"f10a8b3c-1033-4a14-a886-7b3a1503a92f\") " Jul 15 10:21:57.525058 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.525017 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f10a8b3c-1033-4a14-a886-7b3a1503a92f" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:57.525256 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.525229 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f10a8b3c-1033-4a14-a886-7b3a1503a92f" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:57.526965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.526925 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f10a8b3c-1033-4a14-a886-7b3a1503a92f" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:21:57.527095 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.527028 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-kube-api-access-zg5gm" (OuterVolumeSpecName: "kube-api-access-zg5gm") pod "f10a8b3c-1033-4a14-a886-7b3a1503a92f" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f"). InnerVolumeSpecName "kube-api-access-zg5gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:21:57.527223 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.527193 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f10a8b3c-1033-4a14-a886-7b3a1503a92f" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:21:57.527401 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.527381 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f10a8b3c-1033-4a14-a886-7b3a1503a92f" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:21:57.527464 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.527379 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f10a8b3c-1033-4a14-a886-7b3a1503a92f" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:21:57.532840 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.532813 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10a8b3c-1033-4a14-a886-7b3a1503a92f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f10a8b3c-1033-4a14-a886-7b3a1503a92f" (UID: "f10a8b3c-1033-4a14-a886-7b3a1503a92f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:21:57.625763 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.625731 2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-certificates\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:57.625763 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.625762 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f10a8b3c-1033-4a14-a886-7b3a1503a92f-trusted-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:57.625763 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.625772 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-registry-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:57.625919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.625780 2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-image-registry-private-configuration\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:57.625919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.625790 2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-bound-sa-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:57.625919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.625798 2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f10a8b3c-1033-4a14-a886-7b3a1503a92f-ca-trust-extracted\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:57.625919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.625805 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg5gm\" (UniqueName: \"kubernetes.io/projected/f10a8b3c-1033-4a14-a886-7b3a1503a92f-kube-api-access-zg5gm\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:57.625919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.625813 2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f10a8b3c-1033-4a14-a886-7b3a1503a92f-installation-pull-secrets\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:57.642939 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.642914 2585 generic.go:356] "Generic (PLEG): container finished" podID="f10a8b3c-1033-4a14-a886-7b3a1503a92f" containerID="1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b" exitCode=0 Jul 15 10:21:57.643032 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.642957 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" Jul 15 10:21:57.643032 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.643000 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" event={"ID":"f10a8b3c-1033-4a14-a886-7b3a1503a92f","Type":"ContainerDied","Data":"1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b"} Jul 15 10:21:57.643154 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.643050 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bf4bcd5c9-drqg5" event={"ID":"f10a8b3c-1033-4a14-a886-7b3a1503a92f","Type":"ContainerDied","Data":"0aefeb3c17f6bbf8e44185c9ceb234f72784b3035fa7bcd12435afa4586968d1"} Jul 15 10:21:57.643154 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.643091 2585 scope.go:117] "RemoveContainer" containerID="1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b" Jul 15 10:21:57.651154 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.651139 2585 scope.go:117] "RemoveContainer" containerID="1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b" Jul 15 10:21:57.651394 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:21:57.651375 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b\": container with ID starting with 1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b not found: ID does not exist" containerID="1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b" Jul 15 10:21:57.651461 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.651401 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b"} err="failed to get container status \"1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b\": rpc error: code = NotFound desc = could not find container \"1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b\": container with ID starting with 1d33f4190778e05a45483618583687861ba38d6cde27e821d7b75feabe354c4b not found: ID does not exist" Jul 15 10:21:57.665003 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.664985 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bf4bcd5c9-drqg5"] Jul 15 10:21:57.669682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:57.669664 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bf4bcd5c9-drqg5"] Jul 15 10:21:58.933177 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:58.933143 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f10a8b3c-1033-4a14-a886-7b3a1503a92f" path="/var/lib/kubelet/pods/f10a8b3c-1033-4a14-a886-7b3a1503a92f/volumes" Jul 15 10:21:59.324334 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.324298 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-675b5db768-wb7m7" podUID="93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" containerName="console" containerID="cri-o://c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b" gracePeriod=15 Jul 15 10:21:59.561292 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.561270 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:59.641249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641184 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvfbh\" (UniqueName: \"kubernetes.io/projected/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-kube-api-access-xvfbh\") pod \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " Jul 15 10:21:59.641249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641222 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-serving-cert\") pod \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " Jul 15 10:21:59.641422 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641257 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-oauth-config\") pod \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " Jul 15 10:21:59.641422 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641287 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-service-ca\") pod \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " Jul 15 10:21:59.641422 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641365 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-config\") pod \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " Jul 15 10:21:59.641422 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641397 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-oauth-serving-cert\") pod \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " Jul 15 10:21:59.641613 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641458 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-trusted-ca-bundle\") pod \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\" (UID: \"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f\") " Jul 15 10:21:59.641769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641729 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-service-ca" (OuterVolumeSpecName: "service-ca") pod "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" (UID: "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:59.641769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641756 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" (UID: "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:59.642006 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.641769 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-config" (OuterVolumeSpecName: "console-config") pod "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" (UID: "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:59.642049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.642011 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" (UID: "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:21:59.643362 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.643340 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" (UID: "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:21:59.643362 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.643352 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-kube-api-access-xvfbh" (OuterVolumeSpecName: "kube-api-access-xvfbh") pod "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" (UID: "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f"). InnerVolumeSpecName "kube-api-access-xvfbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:21:59.643478 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.643357 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" (UID: "93abd7c8-e1d4-440f-b1b8-ba7511e0f34f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:21:59.654578 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.654554 2585 generic.go:356] "Generic (PLEG): container finished" podID="93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" containerID="c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b" exitCode=0 Jul 15 10:21:59.654657 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.654599 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675b5db768-wb7m7" Jul 15 10:21:59.654657 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.654631 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675b5db768-wb7m7" event={"ID":"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f","Type":"ContainerDied","Data":"c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b"} Jul 15 10:21:59.654723 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.654682 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675b5db768-wb7m7" event={"ID":"93abd7c8-e1d4-440f-b1b8-ba7511e0f34f","Type":"ContainerDied","Data":"74baf3bae3acd597419fde0ab97baa1afdbbfab2b11203e5da6352f971e4765a"} Jul 15 10:21:59.654723 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.654703 2585 scope.go:117] "RemoveContainer" containerID="c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b" Jul 15 10:21:59.663612 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.663592 2585 scope.go:117] "RemoveContainer" containerID="c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b" Jul 15 10:21:59.663848 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:21:59.663829 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b\": container with ID starting with c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b not found: ID does not exist" containerID="c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b" Jul 15 10:21:59.663914 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.663859 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b"} err="failed to get container status \"c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b\": rpc error: code = NotFound desc = could not find container \"c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b\": container with ID starting with c62a04f24cd3381c8d0d05ea8df56c73e476a20306deb76cd85c328ccb0f869b not found: ID does not exist" Jul 15 10:21:59.676600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.676567 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-675b5db768-wb7m7"] Jul 15 10:21:59.679405 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.679386 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-675b5db768-wb7m7"] Jul 15 10:21:59.742289 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.742264 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:59.742289 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.742287 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-oauth-serving-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:59.742419 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.742301 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-trusted-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:59.742419 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.742324 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvfbh\" (UniqueName: \"kubernetes.io/projected/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-kube-api-access-xvfbh\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:59.742419 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.742338 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-serving-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:59.742419 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.742351 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-console-oauth-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:21:59.742419 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:21:59.742363 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f-service-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:00.660158 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:00.660094 2585 generic.go:356] "Generic (PLEG): container finished" podID="5d2eab90-c6a8-4554-b303-102ede8d8570" containerID="3e3064a5ba29a3a3dfce03e9e8f85d580532e97d2517a8c51d7c7e156ff08228" exitCode=0 Jul 15 10:22:00.660524 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:00.660180 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" event={"ID":"5d2eab90-c6a8-4554-b303-102ede8d8570","Type":"ContainerDied","Data":"3e3064a5ba29a3a3dfce03e9e8f85d580532e97d2517a8c51d7c7e156ff08228"} Jul 15 10:22:00.660590 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:00.660524 2585 scope.go:117] "RemoveContainer" containerID="3e3064a5ba29a3a3dfce03e9e8f85d580532e97d2517a8c51d7c7e156ff08228" Jul 15 10:22:00.932634 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:00.932546 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" path="/var/lib/kubelet/pods/93abd7c8-e1d4-440f-b1b8-ba7511e0f34f/volumes" Jul 15 10:22:01.667217 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:01.667182 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-6nkd9" event={"ID":"5d2eab90-c6a8-4554-b303-102ede8d8570","Type":"ContainerStarted","Data":"c7fa69d830e43a67db3da81f6d96cf562f58c0b97655d9d6c4de06c7a8cddb7e"} Jul 15 10:22:05.687191 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:05.687163 2585 generic.go:356] "Generic (PLEG): container finished" podID="481a73d5-b917-4f18-84eb-03aa60dc1e1a" containerID="e3fdc45334bf8318f114e9bd670420b31f9795d6d5bab3b1525981fb281452e0" exitCode=0 Jul 15 10:22:05.687562 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:05.687224 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" event={"ID":"481a73d5-b917-4f18-84eb-03aa60dc1e1a","Type":"ContainerDied","Data":"e3fdc45334bf8318f114e9bd670420b31f9795d6d5bab3b1525981fb281452e0"} Jul 15 10:22:05.687562 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:05.687498 2585 scope.go:117] "RemoveContainer" containerID="e3fdc45334bf8318f114e9bd670420b31f9795d6d5bab3b1525981fb281452e0" Jul 15 10:22:06.693281 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:06.693247 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-t4dcj" event={"ID":"481a73d5-b917-4f18-84eb-03aa60dc1e1a","Type":"ContainerStarted","Data":"abde7f070ef0b19150aa45467322f80f887cf174653eaa8271c83858765a48db"} Jul 15 10:22:30.698718 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:30.698676 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:30.715988 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:30.715961 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:30.812973 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:30.812949 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:44.602368 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.602336 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 10:22:44.602855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.602734 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="alertmanager" containerID="cri-o://841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2" gracePeriod=120 Jul 15 10:22:44.602855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.602776 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy-metric" containerID="cri-o://e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383" gracePeriod=120 Jul 15 10:22:44.602981 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.602846 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="config-reloader" containerID="cri-o://2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e" gracePeriod=120 Jul 15 10:22:44.602981 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.602848 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy" containerID="cri-o://a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6" gracePeriod=120 Jul 15 10:22:44.603111 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.602851 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy-web" containerID="cri-o://969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3" gracePeriod=120 Jul 15 10:22:44.603111 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.602856 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="prom-label-proxy" containerID="cri-o://a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c" gracePeriod=120 Jul 15 10:22:44.859092 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.859008 2585 generic.go:356] "Generic (PLEG): container finished" podID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerID="a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c" exitCode=0 Jul 15 10:22:44.859092 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.859033 2585 generic.go:356] "Generic (PLEG): container finished" podID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerID="a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6" exitCode=0 Jul 15 10:22:44.859092 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.859041 2585 generic.go:356] "Generic (PLEG): container finished" podID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerID="2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e" exitCode=0 Jul 15 10:22:44.859092 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.859046 2585 generic.go:356] "Generic (PLEG): container finished" podID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerID="841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2" exitCode=0 Jul 15 10:22:44.859342 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.859091 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerDied","Data":"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c"} Jul 15 10:22:44.859342 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.859181 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerDied","Data":"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6"} Jul 15 10:22:44.859342 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.859224 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerDied","Data":"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e"} Jul 15 10:22:44.859342 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:44.859265 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerDied","Data":"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2"} Jul 15 10:22:45.842160 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.842139 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:22:45.865851 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.865830 2585 generic.go:356] "Generic (PLEG): container finished" podID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerID="e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383" exitCode=0 Jul 15 10:22:45.865851 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.865850 2585 generic.go:356] "Generic (PLEG): container finished" podID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerID="969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3" exitCode=0 Jul 15 10:22:45.865984 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.865897 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 10:22:45.865984 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.865908 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerDied","Data":"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383"} Jul 15 10:22:45.865984 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.865976 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerDied","Data":"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3"} Jul 15 10:22:45.866097 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.866004 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f17b7dd-03fe-4e31-818e-6513b4e5baac","Type":"ContainerDied","Data":"a6e78dbe8d6ebc234d5307c1eafb808407602d6f2f6b1dff9dbb146319002f83"} Jul 15 10:22:45.866097 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.866040 2585 scope.go:117] "RemoveContainer" containerID="a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c" Jul 15 10:22:45.872749 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.872732 2585 scope.go:117] "RemoveContainer" containerID="e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383" Jul 15 10:22:45.879372 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.879355 2585 scope.go:117] "RemoveContainer" containerID="a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6" Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.881801 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-web\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.881846 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-main-db\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.881873 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.881913 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.881945 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-cluster-tls-config\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.881973 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl8bs\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-kube-api-access-wl8bs\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.882006 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-volume\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.882042 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-web-config\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.882103 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.882131 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.882153 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-tls-assets\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.882693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.882179 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-out\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.883545 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.883108 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:45.883753 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.883714 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:22:45.885503 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.885466 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:45.888111 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.886303 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-kube-api-access-wl8bs" (OuterVolumeSpecName: "kube-api-access-wl8bs") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "kube-api-access-wl8bs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:22:45.888111 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.886413 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:45.889245 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.889128 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:45.889417 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.889355 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-out" (OuterVolumeSpecName: "config-out") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:22:45.890321 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.890278 2585 scope.go:117] "RemoveContainer" containerID="969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3" Jul 15 10:22:45.893913 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.893878 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:22:45.894358 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.894333 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-volume" (OuterVolumeSpecName: "config-volume") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:45.894830 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.894806 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:45.896469 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.896447 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:45.899900 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.899882 2585 scope.go:117] "RemoveContainer" containerID="2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e" Jul 15 10:22:45.905008 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.904959 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-web-config" (OuterVolumeSpecName: "web-config") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:45.906627 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.906609 2585 scope.go:117] "RemoveContainer" containerID="841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2" Jul 15 10:22:45.915163 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.915146 2585 scope.go:117] "RemoveContainer" containerID="a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b" Jul 15 10:22:45.922332 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.922315 2585 scope.go:117] "RemoveContainer" containerID="a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c" Jul 15 10:22:45.922581 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:45.922563 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c\": container with ID starting with a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c not found: ID does not exist" containerID="a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c" Jul 15 10:22:45.922658 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.922592 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c"} err="failed to get container status \"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c\": rpc error: code = NotFound desc = could not find container \"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c\": container with ID starting with a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c not found: ID does not exist" Jul 15 10:22:45.922658 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.922616 2585 scope.go:117] "RemoveContainer" containerID="e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383" Jul 15 10:22:45.922851 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:45.922831 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383\": container with ID starting with e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383 not found: ID does not exist" containerID="e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383" Jul 15 10:22:45.922889 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.922858 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383"} err="failed to get container status \"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383\": rpc error: code = NotFound desc = could not find container \"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383\": container with ID starting with e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383 not found: ID does not exist" Jul 15 10:22:45.922889 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.922873 2585 scope.go:117] "RemoveContainer" containerID="a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6" Jul 15 10:22:45.923138 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:45.923113 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6\": container with ID starting with a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6 not found: ID does not exist" containerID="a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6" Jul 15 10:22:45.923222 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.923149 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6"} err="failed to get container status \"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6\": rpc error: code = NotFound desc = could not find container \"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6\": container with ID starting with a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6 not found: ID does not exist" Jul 15 10:22:45.923222 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.923173 2585 scope.go:117] "RemoveContainer" containerID="969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3" Jul 15 10:22:45.923431 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:45.923413 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3\": container with ID starting with 969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3 not found: ID does not exist" containerID="969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3" Jul 15 10:22:45.923498 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.923435 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3"} err="failed to get container status \"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3\": rpc error: code = NotFound desc = could not find container \"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3\": container with ID starting with 969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3 not found: ID does not exist" Jul 15 10:22:45.923498 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.923450 2585 scope.go:117] "RemoveContainer" containerID="2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e" Jul 15 10:22:45.923666 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:45.923649 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e\": container with ID starting with 2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e not found: ID does not exist" containerID="2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e" Jul 15 10:22:45.923726 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.923669 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e"} err="failed to get container status \"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e\": rpc error: code = NotFound desc = could not find container \"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e\": container with ID starting with 2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e not found: ID does not exist" Jul 15 10:22:45.923726 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.923683 2585 scope.go:117] "RemoveContainer" containerID="841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2" Jul 15 10:22:45.923904 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:45.923885 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2\": container with ID starting with 841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2 not found: ID does not exist" containerID="841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2" Jul 15 10:22:45.923958 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.923906 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2"} err="failed to get container status \"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2\": rpc error: code = NotFound desc = could not find container \"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2\": container with ID starting with 841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2 not found: ID does not exist" Jul 15 10:22:45.923958 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.923917 2585 scope.go:117] "RemoveContainer" containerID="a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b" Jul 15 10:22:45.924142 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:45.924121 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b\": container with ID starting with a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b not found: ID does not exist" containerID="a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b" Jul 15 10:22:45.924207 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924147 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b"} err="failed to get container status \"a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b\": rpc error: code = NotFound desc = could not find container \"a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b\": container with ID starting with a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b not found: ID does not exist" Jul 15 10:22:45.924207 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924161 2585 scope.go:117] "RemoveContainer" containerID="a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c" Jul 15 10:22:45.924343 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924328 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c"} err="failed to get container status \"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c\": rpc error: code = NotFound desc = could not find container \"a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c\": container with ID starting with a4e3713a7d7f2c1ae338fba81baed0067105ae1bc2d7a93684fd61c131c9274c not found: ID does not exist" Jul 15 10:22:45.924400 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924342 2585 scope.go:117] "RemoveContainer" containerID="e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383" Jul 15 10:22:45.924512 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924497 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383"} err="failed to get container status \"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383\": rpc error: code = NotFound desc = could not find container \"e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383\": container with ID starting with e07de211c905d16fb6334931d40faa95d4627ee20983f0b7c74f7ce3dda89383 not found: ID does not exist" Jul 15 10:22:45.924577 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924512 2585 scope.go:117] "RemoveContainer" containerID="a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6" Jul 15 10:22:45.924682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924664 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6"} err="failed to get container status \"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6\": rpc error: code = NotFound desc = could not find container \"a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6\": container with ID starting with a70a83660436956cb425e9a4715ca161ab9e191534a7af83ba80608b75957ad6 not found: ID does not exist" Jul 15 10:22:45.924743 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924682 2585 scope.go:117] "RemoveContainer" containerID="969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3" Jul 15 10:22:45.924856 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924841 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3"} err="failed to get container status \"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3\": rpc error: code = NotFound desc = could not find container \"969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3\": container with ID starting with 969970c43b80fa1e17cfeae6e55661a741b7537f270a09e564a0d2d9229a0df3 not found: ID does not exist" Jul 15 10:22:45.924914 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.924856 2585 scope.go:117] "RemoveContainer" containerID="2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e" Jul 15 10:22:45.925034 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.925007 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e"} err="failed to get container status \"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e\": rpc error: code = NotFound desc = could not find container \"2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e\": container with ID starting with 2e91d76427b222c645a10736a360dfbd62890e8b3b816d4ee87fd372e485774e not found: ID does not exist" Jul 15 10:22:45.925165 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.925036 2585 scope.go:117] "RemoveContainer" containerID="841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2" Jul 15 10:22:45.925250 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.925235 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2"} err="failed to get container status \"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2\": rpc error: code = NotFound desc = could not find container \"841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2\": container with ID starting with 841240af7003efea062a91110a6e22da74e17eb3eca4f8baad24beb4c1c12ee2 not found: ID does not exist" Jul 15 10:22:45.925304 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.925250 2585 scope.go:117] "RemoveContainer" containerID="a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b" Jul 15 10:22:45.925485 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.925466 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b"} err="failed to get container status \"a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b\": rpc error: code = NotFound desc = could not find container \"a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b\": container with ID starting with a6c04af1bc165b309d8f772e9722ec469cfb7d96379e6fd5c4902c6d5d1ae08b not found: ID does not exist" Jul 15 10:22:45.982693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982671 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-metrics-client-ca\") pod \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\" (UID: \"2f17b7dd-03fe-4e31-818e-6513b4e5baac\") " Jul 15 10:22:45.982806 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982794 2585 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-cluster-tls-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982809 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl8bs\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-kube-api-access-wl8bs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982818 2585 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982831 2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-web-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982849 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982843 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982853 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-main-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982861 2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f17b7dd-03fe-4e31-818e-6513b4e5baac-tls-assets\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982869 2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-config-out\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982876 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982887 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f17b7dd-03fe-4e31-818e-6513b4e5baac-alertmanager-main-db\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982895 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982905 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f17b7dd-03fe-4e31-818e-6513b4e5baac-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:45.982964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:45.982936 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2f17b7dd-03fe-4e31-818e-6513b4e5baac" (UID: "2f17b7dd-03fe-4e31-818e-6513b4e5baac"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:46.084006 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:46.083984 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f17b7dd-03fe-4e31-818e-6513b4e5baac-metrics-client-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:46.192797 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:46.192774 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 10:22:46.198415 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:46.198396 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 10:22:46.932661 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:46.932593 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" path="/var/lib/kubelet/pods/2f17b7dd-03fe-4e31-818e-6513b4e5baac/volumes" Jul 15 10:22:48.842121 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:48.842086 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:22:48.842513 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:48.842373 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="prometheus" containerID="cri-o://ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" gracePeriod=600 Jul 15 10:22:48.842513 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:48.842383 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy" containerID="cri-o://c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" gracePeriod=600 Jul 15 10:22:48.842513 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:48.842417 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" gracePeriod=600 Jul 15 10:22:48.842513 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:48.842427 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy-web" containerID="cri-o://08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" gracePeriod=600 Jul 15 10:22:48.842513 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:48.842461 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="config-reloader" containerID="cri-o://48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" gracePeriod=600 Jul 15 10:22:48.842513 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:48.842396 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="thanos-sidecar" containerID="cri-o://d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" gracePeriod=600 Jul 15 10:22:49.091752 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.091729 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:49.107226 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107172 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107226 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107217 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107354 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107244 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-tls-assets\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107354 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107280 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-1\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107354 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107324 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-metrics-client-certs\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107354 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107347 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-db\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107380 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-thanos-prometheus-http-client-file\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107413 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djw8g\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-kube-api-access-djw8g\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107439 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-metrics-client-ca\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107466 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-grpc-tls\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107489 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-serving-certs-ca-bundle\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107514 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-tls\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107555 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-kubelet-serving-ca-bundle\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107597 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-config\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107620 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-0\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107648 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-web-config\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107673 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-kube-rbac-proxy\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107716 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-2\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107741 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-trusted-ca-bundle\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.107878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107773 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-config-out\") pod \"5ea1fa78-536e-456d-9083-cc9019fe40b8\" (UID: \"5ea1fa78-536e-456d-9083-cc9019fe40b8\") " Jul 15 10:22:49.108738 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107639 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:49.108835 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107870 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:49.108835 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.107945 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:49.108955 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.108887 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:49.109181 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.108998 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:49.109418 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.109389 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:49.110497 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.110464 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:22:49.110595 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.110558 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:22:49.110979 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.110952 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.111080 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.110954 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-kube-api-access-djw8g" (OuterVolumeSpecName: "kube-api-access-djw8g") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "kube-api-access-djw8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:22:49.113569 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.113526 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.113855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.113755 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.113855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.113778 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.113855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.113825 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:22:49.113855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.113838 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.114637 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.114331 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.114637 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.114335 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-config-out" (OuterVolumeSpecName: "config-out") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:22:49.114637 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.114382 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-config" (OuterVolumeSpecName: "config") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.114989 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.114966 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.125697 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.125634 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-web-config" (OuterVolumeSpecName: "web-config") pod "5ea1fa78-536e-456d-9083-cc9019fe40b8" (UID: "5ea1fa78-536e-456d-9083-cc9019fe40b8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:22:49.208437 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208404 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208437 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208432 2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-tls-assets\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208437 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208443 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208452 2585 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-metrics-client-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208460 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-db\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208470 2585 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208479 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djw8g\" (UniqueName: \"kubernetes.io/projected/5ea1fa78-536e-456d-9083-cc9019fe40b8-kube-api-access-djw8g\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208488 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-metrics-client-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208496 2585 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-grpc-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208505 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208514 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208522 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208530 2585 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208537 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208545 2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-web-config\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208553 2585 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-kube-rbac-proxy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208560 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208569 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea1fa78-536e-456d-9083-cc9019fe40b8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208576 2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea1fa78-536e-456d-9083-cc9019fe40b8-config-out\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.208639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.208584 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5ea1fa78-536e-456d-9083-cc9019fe40b8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:22:49.892352 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892318 2585 generic.go:356] "Generic (PLEG): container finished" podID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" exitCode=0 Jul 15 10:22:49.892352 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892349 2585 generic.go:356] "Generic (PLEG): container finished" podID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" exitCode=0 Jul 15 10:22:49.892352 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892360 2585 generic.go:356] "Generic (PLEG): container finished" podID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" exitCode=0 Jul 15 10:22:49.892352 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892368 2585 generic.go:356] "Generic (PLEG): container finished" podID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" exitCode=0 Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892376 2585 generic.go:356] "Generic (PLEG): container finished" podID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" exitCode=0 Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892384 2585 generic.go:356] "Generic (PLEG): container finished" podID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" exitCode=0 Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892394 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerDied","Data":"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079"} Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892423 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892498 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerDied","Data":"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db"} Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892549 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerDied","Data":"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0"} Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892574 2585 scope.go:117] "RemoveContainer" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892587 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerDied","Data":"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a"} Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892625 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerDied","Data":"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4"} Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892662 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerDied","Data":"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e"} Jul 15 10:22:49.892893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.892701 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5ea1fa78-536e-456d-9083-cc9019fe40b8","Type":"ContainerDied","Data":"316668b481a83557b198250592cb5d3ec6b54a80e1f6734f19d6a731ee09fd4a"} Jul 15 10:22:49.901385 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.901229 2585 scope.go:117] "RemoveContainer" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" Jul 15 10:22:49.908533 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.908516 2585 scope.go:117] "RemoveContainer" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" Jul 15 10:22:49.915123 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.915100 2585 scope.go:117] "RemoveContainer" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" Jul 15 10:22:49.920810 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.920788 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:22:49.922150 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.922135 2585 scope.go:117] "RemoveContainer" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" Jul 15 10:22:49.926300 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.926280 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:22:49.928930 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.928916 2585 scope.go:117] "RemoveContainer" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" Jul 15 10:22:49.935482 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.935468 2585 scope.go:117] "RemoveContainer" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" Jul 15 10:22:49.941658 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.941645 2585 scope.go:117] "RemoveContainer" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" Jul 15 10:22:49.941888 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:49.941871 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": container with ID starting with aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079 not found: ID does not exist" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" Jul 15 10:22:49.941930 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.941894 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079"} err="failed to get container status \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": rpc error: code = NotFound desc = could not find container \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": container with ID starting with aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079 not found: ID does not exist" Jul 15 10:22:49.941930 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.941909 2585 scope.go:117] "RemoveContainer" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" Jul 15 10:22:49.942163 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:49.942144 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": container with ID starting with c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db not found: ID does not exist" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" Jul 15 10:22:49.942235 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.942173 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db"} err="failed to get container status \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": rpc error: code = NotFound desc = could not find container \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": container with ID starting with c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db not found: ID does not exist" Jul 15 10:22:49.942235 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.942197 2585 scope.go:117] "RemoveContainer" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" Jul 15 10:22:49.942402 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:49.942375 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": container with ID starting with 08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0 not found: ID does not exist" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" Jul 15 10:22:49.942446 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.942397 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0"} err="failed to get container status \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": rpc error: code = NotFound desc = could not find container \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": container with ID starting with 08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0 not found: ID does not exist" Jul 15 10:22:49.942446 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.942416 2585 scope.go:117] "RemoveContainer" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" Jul 15 10:22:49.942648 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:49.942630 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": container with ID starting with d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a not found: ID does not exist" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" Jul 15 10:22:49.942723 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.942655 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a"} err="failed to get container status \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": rpc error: code = NotFound desc = could not find container \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": container with ID starting with d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a not found: ID does not exist" Jul 15 10:22:49.942723 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.942676 2585 scope.go:117] "RemoveContainer" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" Jul 15 10:22:49.942913 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:49.942895 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": container with ID starting with 48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4 not found: ID does not exist" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" Jul 15 10:22:49.942957 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.942919 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4"} err="failed to get container status \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": rpc error: code = NotFound desc = could not find container \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": container with ID starting with 48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4 not found: ID does not exist" Jul 15 10:22:49.942957 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.942933 2585 scope.go:117] "RemoveContainer" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" Jul 15 10:22:49.943186 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:49.943162 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": container with ID starting with ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e not found: ID does not exist" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" Jul 15 10:22:49.943238 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.943189 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e"} err="failed to get container status \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": rpc error: code = NotFound desc = could not find container \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": container with ID starting with ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e not found: ID does not exist" Jul 15 10:22:49.943238 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.943206 2585 scope.go:117] "RemoveContainer" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" Jul 15 10:22:49.943416 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:22:49.943401 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": container with ID starting with 5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d not found: ID does not exist" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" Jul 15 10:22:49.943455 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.943418 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d"} err="failed to get container status \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": rpc error: code = NotFound desc = could not find container \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": container with ID starting with 5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d not found: ID does not exist" Jul 15 10:22:49.943455 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.943442 2585 scope.go:117] "RemoveContainer" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" Jul 15 10:22:49.943647 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.943632 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079"} err="failed to get container status \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": rpc error: code = NotFound desc = could not find container \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": container with ID starting with aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079 not found: ID does not exist" Jul 15 10:22:49.943693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.943647 2585 scope.go:117] "RemoveContainer" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" Jul 15 10:22:49.943889 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.943867 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db"} err="failed to get container status \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": rpc error: code = NotFound desc = could not find container \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": container with ID starting with c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db not found: ID does not exist" Jul 15 10:22:49.943950 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.943889 2585 scope.go:117] "RemoveContainer" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" Jul 15 10:22:49.944181 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.944161 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0"} err="failed to get container status \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": rpc error: code = NotFound desc = could not find container \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": container with ID starting with 08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0 not found: ID does not exist" Jul 15 10:22:49.944260 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.944179 2585 scope.go:117] "RemoveContainer" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" Jul 15 10:22:49.944441 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.944423 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a"} err="failed to get container status \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": rpc error: code = NotFound desc = could not find container \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": container with ID starting with d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a not found: ID does not exist" Jul 15 10:22:49.944496 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.944442 2585 scope.go:117] "RemoveContainer" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" Jul 15 10:22:49.944665 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.944648 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4"} err="failed to get container status \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": rpc error: code = NotFound desc = could not find container \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": container with ID starting with 48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4 not found: ID does not exist" Jul 15 10:22:49.944710 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.944666 2585 scope.go:117] "RemoveContainer" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" Jul 15 10:22:49.944848 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.944832 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e"} err="failed to get container status \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": rpc error: code = NotFound desc = could not find container \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": container with ID starting with ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e not found: ID does not exist" Jul 15 10:22:49.944896 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.944850 2585 scope.go:117] "RemoveContainer" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" Jul 15 10:22:49.945034 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945019 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d"} err="failed to get container status \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": rpc error: code = NotFound desc = could not find container \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": container with ID starting with 5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d not found: ID does not exist" Jul 15 10:22:49.945102 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945034 2585 scope.go:117] "RemoveContainer" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" Jul 15 10:22:49.945269 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945253 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079"} err="failed to get container status \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": rpc error: code = NotFound desc = could not find container \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": container with ID starting with aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079 not found: ID does not exist" Jul 15 10:22:49.945318 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945269 2585 scope.go:117] "RemoveContainer" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" Jul 15 10:22:49.945497 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945479 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db"} err="failed to get container status \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": rpc error: code = NotFound desc = could not find container \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": container with ID starting with c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db not found: ID does not exist" Jul 15 10:22:49.945497 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945496 2585 scope.go:117] "RemoveContainer" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" Jul 15 10:22:49.945700 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945684 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0"} err="failed to get container status \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": rpc error: code = NotFound desc = could not find container \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": container with ID starting with 08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0 not found: ID does not exist" Jul 15 10:22:49.945700 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945700 2585 scope.go:117] "RemoveContainer" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" Jul 15 10:22:49.945921 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945906 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a"} err="failed to get container status \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": rpc error: code = NotFound desc = could not find container \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": container with ID starting with d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a not found: ID does not exist" Jul 15 10:22:49.945921 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.945919 2585 scope.go:117] "RemoveContainer" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" Jul 15 10:22:49.946122 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946106 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4"} err="failed to get container status \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": rpc error: code = NotFound desc = could not find container \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": container with ID starting with 48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4 not found: ID does not exist" Jul 15 10:22:49.946169 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946129 2585 scope.go:117] "RemoveContainer" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" Jul 15 10:22:49.946317 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946302 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e"} err="failed to get container status \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": rpc error: code = NotFound desc = could not find container \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": container with ID starting with ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e not found: ID does not exist" Jul 15 10:22:49.946317 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946317 2585 scope.go:117] "RemoveContainer" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" Jul 15 10:22:49.946489 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946475 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d"} err="failed to get container status \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": rpc error: code = NotFound desc = could not find container \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": container with ID starting with 5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d not found: ID does not exist" Jul 15 10:22:49.946530 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946489 2585 scope.go:117] "RemoveContainer" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" Jul 15 10:22:49.946693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946676 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079"} err="failed to get container status \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": rpc error: code = NotFound desc = could not find container \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": container with ID starting with aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079 not found: ID does not exist" Jul 15 10:22:49.946693 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946692 2585 scope.go:117] "RemoveContainer" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" Jul 15 10:22:49.946907 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946889 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db"} err="failed to get container status \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": rpc error: code = NotFound desc = could not find container \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": container with ID starting with c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db not found: ID does not exist" Jul 15 10:22:49.946965 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.946908 2585 scope.go:117] "RemoveContainer" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" Jul 15 10:22:49.947129 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947108 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0"} err="failed to get container status \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": rpc error: code = NotFound desc = could not find container \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": container with ID starting with 08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0 not found: ID does not exist" Jul 15 10:22:49.947129 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947129 2585 scope.go:117] "RemoveContainer" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" Jul 15 10:22:49.947335 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947319 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a"} err="failed to get container status \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": rpc error: code = NotFound desc = could not find container \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": container with ID starting with d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a not found: ID does not exist" Jul 15 10:22:49.947377 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947335 2585 scope.go:117] "RemoveContainer" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" Jul 15 10:22:49.947553 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947534 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4"} err="failed to get container status \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": rpc error: code = NotFound desc = could not find container \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": container with ID starting with 48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4 not found: ID does not exist" Jul 15 10:22:49.947609 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947553 2585 scope.go:117] "RemoveContainer" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" Jul 15 10:22:49.947733 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947719 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e"} err="failed to get container status \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": rpc error: code = NotFound desc = could not find container \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": container with ID starting with ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e not found: ID does not exist" Jul 15 10:22:49.947768 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947734 2585 scope.go:117] "RemoveContainer" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" Jul 15 10:22:49.947896 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947881 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d"} err="failed to get container status \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": rpc error: code = NotFound desc = could not find container \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": container with ID starting with 5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d not found: ID does not exist" Jul 15 10:22:49.947935 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.947896 2585 scope.go:117] "RemoveContainer" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" Jul 15 10:22:49.948097 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948051 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079"} err="failed to get container status \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": rpc error: code = NotFound desc = could not find container \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": container with ID starting with aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079 not found: ID does not exist" Jul 15 10:22:49.948097 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948085 2585 scope.go:117] "RemoveContainer" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" Jul 15 10:22:49.948297 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948282 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db"} err="failed to get container status \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": rpc error: code = NotFound desc = could not find container \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": container with ID starting with c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db not found: ID does not exist" Jul 15 10:22:49.948342 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948298 2585 scope.go:117] "RemoveContainer" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" Jul 15 10:22:49.948490 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948464 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0"} err="failed to get container status \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": rpc error: code = NotFound desc = could not find container \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": container with ID starting with 08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0 not found: ID does not exist" Jul 15 10:22:49.948530 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948491 2585 scope.go:117] "RemoveContainer" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" Jul 15 10:22:49.948689 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948669 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a"} err="failed to get container status \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": rpc error: code = NotFound desc = could not find container \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": container with ID starting with d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a not found: ID does not exist" Jul 15 10:22:49.948726 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948689 2585 scope.go:117] "RemoveContainer" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" Jul 15 10:22:49.948885 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948863 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4"} err="failed to get container status \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": rpc error: code = NotFound desc = could not find container \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": container with ID starting with 48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4 not found: ID does not exist" Jul 15 10:22:49.948885 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.948884 2585 scope.go:117] "RemoveContainer" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" Jul 15 10:22:49.949095 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.949076 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e"} err="failed to get container status \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": rpc error: code = NotFound desc = could not find container \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": container with ID starting with ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e not found: ID does not exist" Jul 15 10:22:49.949095 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.949095 2585 scope.go:117] "RemoveContainer" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" Jul 15 10:22:49.949293 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.949275 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d"} err="failed to get container status \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": rpc error: code = NotFound desc = could not find container \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": container with ID starting with 5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d not found: ID does not exist" Jul 15 10:22:49.949353 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.949296 2585 scope.go:117] "RemoveContainer" containerID="aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079" Jul 15 10:22:49.949555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.949530 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079"} err="failed to get container status \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": rpc error: code = NotFound desc = could not find container \"aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079\": container with ID starting with aa0fcf53b88fa9cbf01c6df050dc7f63234a014507649e5a752beb0b6fcc0079 not found: ID does not exist" Jul 15 10:22:49.949555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.949553 2585 scope.go:117] "RemoveContainer" containerID="c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db" Jul 15 10:22:49.949801 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.949776 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db"} err="failed to get container status \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": rpc error: code = NotFound desc = could not find container \"c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db\": container with ID starting with c5b18abe8ec45ef32586b4b8bce2ce0d31ef42c23c27843b745a55f66ff335db not found: ID does not exist" Jul 15 10:22:49.949857 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.949804 2585 scope.go:117] "RemoveContainer" containerID="08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0" Jul 15 10:22:49.950030 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950006 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0"} err="failed to get container status \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": rpc error: code = NotFound desc = could not find container \"08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0\": container with ID starting with 08f0a9398169ceb14acfd7386f47209a9a0a484901ed9eae97ae4da4f5c428d0 not found: ID does not exist" Jul 15 10:22:49.950030 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950030 2585 scope.go:117] "RemoveContainer" containerID="d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a" Jul 15 10:22:49.950282 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950265 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a"} err="failed to get container status \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": rpc error: code = NotFound desc = could not find container \"d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a\": container with ID starting with d84b66d22819b4371d5e84d346f71c4b456e18be60224a65e19e664a1d0f324a not found: ID does not exist" Jul 15 10:22:49.950330 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950283 2585 scope.go:117] "RemoveContainer" containerID="48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4" Jul 15 10:22:49.950511 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950495 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4"} err="failed to get container status \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": rpc error: code = NotFound desc = could not find container \"48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4\": container with ID starting with 48efac7f809345639a81842daf0dabe0ffcf7ccf0f75146dda4846919e519eb4 not found: ID does not exist" Jul 15 10:22:49.950565 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950511 2585 scope.go:117] "RemoveContainer" containerID="ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e" Jul 15 10:22:49.950691 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950670 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e"} err="failed to get container status \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": rpc error: code = NotFound desc = could not find container \"ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e\": container with ID starting with ccd923cf5736bc615bae6ca78c23006c49e7f0a619c8b1724d374ad419eba41e not found: ID does not exist" Jul 15 10:22:49.950730 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950692 2585 scope.go:117] "RemoveContainer" containerID="5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d" Jul 15 10:22:49.950865 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.950848 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d"} err="failed to get container status \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": rpc error: code = NotFound desc = could not find container \"5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d\": container with ID starting with 5da0fa0228cb2d6663fcb241ba4531c44f8cc6e2dbf7ba6109a769dcc593753d not found: ID does not exist" Jul 15 10:22:49.960933 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.960915 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:22:49.961178 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961166 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" containerName="console" Jul 15 10:22:49.961220 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961179 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" containerName="console" Jul 15 10:22:49.961220 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961187 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="prometheus" Jul 15 10:22:49.961220 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961192 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="prometheus" Jul 15 10:22:49.961220 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961199 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="thanos-sidecar" Jul 15 10:22:49.961220 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961204 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="thanos-sidecar" Jul 15 10:22:49.961220 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961212 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="config-reloader" Jul 15 10:22:49.961220 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961218 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="config-reloader" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961226 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy-web" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961231 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy-web" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961238 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy-metric" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961242 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy-metric" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961251 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="init-config-reloader" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961255 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="init-config-reloader" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961262 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="alertmanager" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961267 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="alertmanager" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961272 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="prom-label-proxy" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961277 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="prom-label-proxy" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961283 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="init-config-reloader" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961288 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="init-config-reloader" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961294 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961298 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961304 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="config-reloader" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961309 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="config-reloader" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961314 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy-web" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961319 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy-web" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961326 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961331 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961340 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f10a8b3c-1033-4a14-a886-7b3a1503a92f" containerName="registry" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961344 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f10a8b3c-1033-4a14-a886-7b3a1503a92f" containerName="registry" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961350 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5c8ea1b-25f1-4d12-981c-458129fbc84f" containerName="console" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961355 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5c8ea1b-25f1-4d12-981c-458129fbc84f" containerName="console" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961360 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy-thanos" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961365 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy-thanos" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961407 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy-metric" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961415 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="93abd7c8-e1d4-440f-b1b8-ba7511e0f34f" containerName="console" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961421 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="thanos-sidecar" Jul 15 10:22:49.961412 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961427 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy-web" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961432 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f10a8b3c-1033-4a14-a886-7b3a1503a92f" containerName="registry" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961438 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="prometheus" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961442 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961448 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5c8ea1b-25f1-4d12-981c-458129fbc84f" containerName="console" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961452 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy-thanos" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961457 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="config-reloader" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961462 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="kube-rbac-proxy" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961468 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="alertmanager" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961473 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="kube-rbac-proxy-web" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961478 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f17b7dd-03fe-4e31-818e-6513b4e5baac" containerName="prom-label-proxy" Jul 15 10:22:49.962211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.961482 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" containerName="config-reloader" Jul 15 10:22:49.965461 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.965445 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:49.968172 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.968154 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 15 10:22:49.968355 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.968338 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6rfbq4tghgn9b\"" Jul 15 10:22:49.968543 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.968528 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 15 10:22:49.968543 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.968537 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 15 10:22:49.968665 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.968530 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 15 10:22:49.968665 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.968569 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 15 10:22:49.968665 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.968603 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 15 10:22:49.968927 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.968900 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 15 10:22:49.969053 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.969040 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 15 10:22:49.969254 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.969239 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 15 10:22:49.969310 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.969281 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 15 10:22:49.969359 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.969309 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-dwfcv\"" Jul 15 10:22:49.969359 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.969317 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 15 10:22:49.969496 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.969397 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 15 10:22:49.969640 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.969624 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 15 10:22:49.977765 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.976506 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 15 10:22:49.982001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.981978 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:22:49.982112 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:49.982092 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 15 10:22:50.014122 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014096 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014210 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014126 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014210 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014148 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014210 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014164 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014210 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014186 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014406 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd6b0204-7223-43e1-871c-eac61a998dc8-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014406 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014283 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014406 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014308 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014406 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014337 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014406 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014367 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014405 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014424 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014449 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd6b0204-7223-43e1-871c-eac61a998dc8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014466 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xrxt\" (UniqueName: \"kubernetes.io/projected/dd6b0204-7223-43e1-871c-eac61a998dc8-kube-api-access-6xrxt\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014483 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014498 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-config\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014529 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014564 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014557 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014574 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.014827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.014634 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115305 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115252 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115305 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115279 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115305 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115295 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115478 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115311 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd6b0204-7223-43e1-871c-eac61a998dc8-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115478 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115327 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115568 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115520 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115617 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115570 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115617 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115608 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115639 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115664 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115707 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd6b0204-7223-43e1-871c-eac61a998dc8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115853 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115735 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xrxt\" (UniqueName: \"kubernetes.io/projected/dd6b0204-7223-43e1-871c-eac61a998dc8-kube-api-access-6xrxt\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115853 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115853 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115793 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-config\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115853 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115823 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.115853 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115847 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.116163 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.116163 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115922 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.116163 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115950 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.116163 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115988 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.116163 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.116015 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.116406 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.116283 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.117826 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.116790 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.117826 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.117080 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.117826 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.115950 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.117826 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.117772 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.118168 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.118154 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.118525 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.118501 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.118662 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.118615 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.119498 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.119366 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.119860 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.119825 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.120755 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.120731 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd6b0204-7223-43e1-871c-eac61a998dc8-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.120884 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.120862 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.120945 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.120868 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.120945 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.120903 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.121047 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.121027 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd6b0204-7223-43e1-871c-eac61a998dc8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.121304 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.121288 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-config\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.121562 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.121541 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd6b0204-7223-43e1-871c-eac61a998dc8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.122313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.122292 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd6b0204-7223-43e1-871c-eac61a998dc8-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.126378 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.126325 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xrxt\" (UniqueName: \"kubernetes.io/projected/dd6b0204-7223-43e1-871c-eac61a998dc8-kube-api-access-6xrxt\") pod \"prometheus-k8s-0\" (UID: \"dd6b0204-7223-43e1-871c-eac61a998dc8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.282325 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.282299 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:22:50.417747 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.417725 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 10:22:50.420038 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:22:50.420005 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6b0204_7223_43e1_871c_eac61a998dc8.slice/crio-ed7b018e8639903f6a37bf614831a970bbc9d39ea3f877417cb6d9de4867b5de WatchSource:0}: Error finding container ed7b018e8639903f6a37bf614831a970bbc9d39ea3f877417cb6d9de4867b5de: Status 404 returned error can't find the container with id ed7b018e8639903f6a37bf614831a970bbc9d39ea3f877417cb6d9de4867b5de Jul 15 10:22:50.898360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.898325 2585 generic.go:356] "Generic (PLEG): container finished" podID="dd6b0204-7223-43e1-871c-eac61a998dc8" containerID="3de1d8ee3371cabd8859357fd416e9edc259ec9bc62f671d703f56a3f9121c91" exitCode=0 Jul 15 10:22:50.898689 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.898414 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd6b0204-7223-43e1-871c-eac61a998dc8","Type":"ContainerDied","Data":"3de1d8ee3371cabd8859357fd416e9edc259ec9bc62f671d703f56a3f9121c91"} Jul 15 10:22:50.898689 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.898488 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd6b0204-7223-43e1-871c-eac61a998dc8","Type":"ContainerStarted","Data":"ed7b018e8639903f6a37bf614831a970bbc9d39ea3f877417cb6d9de4867b5de"} Jul 15 10:22:50.934579 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:50.934312 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ea1fa78-536e-456d-9083-cc9019fe40b8" path="/var/lib/kubelet/pods/5ea1fa78-536e-456d-9083-cc9019fe40b8/volumes" Jul 15 10:22:51.910394 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:51.910363 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd6b0204-7223-43e1-871c-eac61a998dc8","Type":"ContainerStarted","Data":"7f7e636b121c0f0d2df34c92c421f20da28f5bc04d0c195ba97bd610d8ca163e"} Jul 15 10:22:51.910722 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:51.910428 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd6b0204-7223-43e1-871c-eac61a998dc8","Type":"ContainerStarted","Data":"00432bbfa580cf685567b6ac7007e13545ef5170cf34cecac25bd967d8f6b717"} Jul 15 10:22:51.910722 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:51.910457 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd6b0204-7223-43e1-871c-eac61a998dc8","Type":"ContainerStarted","Data":"cbe739b17c1c8464397349ec5775e757aaec2677089fca92c257e7d83885438a"} Jul 15 10:22:51.910722 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:51.910484 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd6b0204-7223-43e1-871c-eac61a998dc8","Type":"ContainerStarted","Data":"dcbc5141190020249e358175b22bbabdda79a0df747e02f3ab4e6c4ae9692da7"} Jul 15 10:22:51.910722 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:51.910508 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd6b0204-7223-43e1-871c-eac61a998dc8","Type":"ContainerStarted","Data":"12e6de4354f080dbb16679c0797322920f716d788ecdead0c8d5df742d0fb839"} Jul 15 10:22:51.910722 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:51.910532 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd6b0204-7223-43e1-871c-eac61a998dc8","Type":"ContainerStarted","Data":"b189298e48fb786345c69371e7919f4ed4ed2621f5a756ca25b039ed1bea117b"} Jul 15 10:22:51.940782 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:51.940714 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.923794954 podStartE2EDuration="2.940694845s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.899891ms" isStatefulPod=false podCreationTimestamp="2026-07-15 10:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:22:51.940435088 +0000 UTC m=+175.456427572" watchObservedRunningTime="2026-07-15 10:22:51.940694845 +0000 UTC m=+175.456687332" Jul 15 10:22:55.282404 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:22:55.282369 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:23:50.283253 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:23:50.283198 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:23:50.299528 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:23:50.299503 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:23:51.157382 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:23:51.157354 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 10:24:54.151383 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.151350 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr"] Jul 15 10:24:54.154847 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.154818 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.158683 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.158654 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 10:24:54.158796 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.158772 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xgzvh\"" Jul 15 10:24:54.159115 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.159090 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 10:24:54.159349 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.159323 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 10:24:54.159453 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.159418 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 10:24:54.184300 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.184268 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr"] Jul 15 10:24:54.189196 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.189171 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7881688-dd73-4d4a-bafe-df3cd0219c78-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.189296 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.189220 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqvb7\" (UniqueName: \"kubernetes.io/projected/c7881688-dd73-4d4a-bafe-df3cd0219c78-kube-api-access-kqvb7\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.189296 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.189246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7881688-dd73-4d4a-bafe-df3cd0219c78-manifests\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.189371 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.189293 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7881688-dd73-4d4a-bafe-df3cd0219c78-tmp\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.189371 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.189350 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7881688-dd73-4d4a-bafe-df3cd0219c78-webhook-cert\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.290480 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.290455 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqvb7\" (UniqueName: \"kubernetes.io/projected/c7881688-dd73-4d4a-bafe-df3cd0219c78-kube-api-access-kqvb7\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.290566 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.290492 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7881688-dd73-4d4a-bafe-df3cd0219c78-manifests\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.290566 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.290514 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7881688-dd73-4d4a-bafe-df3cd0219c78-tmp\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.290646 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.290569 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7881688-dd73-4d4a-bafe-df3cd0219c78-webhook-cert\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.290646 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.290609 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7881688-dd73-4d4a-bafe-df3cd0219c78-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.290844 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.290823 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7881688-dd73-4d4a-bafe-df3cd0219c78-manifests\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.292826 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.292794 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7881688-dd73-4d4a-bafe-df3cd0219c78-tmp\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.292907 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.292800 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7881688-dd73-4d4a-bafe-df3cd0219c78-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.292972 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.292957 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7881688-dd73-4d4a-bafe-df3cd0219c78-webhook-cert\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.305914 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.305897 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqvb7\" (UniqueName: \"kubernetes.io/projected/c7881688-dd73-4d4a-bafe-df3cd0219c78-kube-api-access-kqvb7\") pod \"opendatahub-operator-controller-manager-656947bb4-t6ttr\" (UID: \"c7881688-dd73-4d4a-bafe-df3cd0219c78\") " pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.466215 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.466167 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:24:54.621457 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:54.621428 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr"] Jul 15 10:24:54.624820 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:24:54.624792 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7881688_dd73_4d4a_bafe_df3cd0219c78.slice/crio-f9af4d843b9f7a018cb13487c106114b7564299e3834fb896a1f13a6c29b4e50 WatchSource:0}: Error finding container f9af4d843b9f7a018cb13487c106114b7564299e3834fb896a1f13a6c29b4e50: Status 404 returned error can't find the container with id f9af4d843b9f7a018cb13487c106114b7564299e3834fb896a1f13a6c29b4e50 Jul 15 10:24:55.390577 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:55.390504 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" event={"ID":"c7881688-dd73-4d4a-bafe-df3cd0219c78","Type":"ContainerStarted","Data":"f9af4d843b9f7a018cb13487c106114b7564299e3834fb896a1f13a6c29b4e50"} Jul 15 10:24:57.625103 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:57.625057 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:24:57.625103 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:57.625085 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:24:57.637979 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:57.637961 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 10:24:58.406255 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:58.406218 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" event={"ID":"c7881688-dd73-4d4a-bafe-df3cd0219c78","Type":"ContainerStarted","Data":"39498019585e4a74c0cc3f9a0a0a2fbaa92900b7a1917ea3657c42715fcab025"} Jul 15 10:24:59.412619 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:59.412580 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7881688-dd73-4d4a-bafe-df3cd0219c78" containerID="39498019585e4a74c0cc3f9a0a0a2fbaa92900b7a1917ea3657c42715fcab025" exitCode=0 Jul 15 10:24:59.413008 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:59.412636 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" event={"ID":"c7881688-dd73-4d4a-bafe-df3cd0219c78","Type":"ContainerDied","Data":"39498019585e4a74c0cc3f9a0a0a2fbaa92900b7a1917ea3657c42715fcab025"} Jul 15 10:24:59.413201 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:24:59.413181 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 10:25:00.418626 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:00.418585 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" event={"ID":"c7881688-dd73-4d4a-bafe-df3cd0219c78","Type":"ContainerStarted","Data":"76528ae754ba8bf9927d9fe868475ad3c04b923f9a8253f3ac1c5a2f30ac9bf8"} Jul 15 10:25:00.419019 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:00.418689 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:25:00.459132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:00.459053 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" podStartSLOduration=1.991117941 podStartE2EDuration="6.459034724s" totalImagesPullingTime="3.466681168s" totalInitContainerRuntime="1.001235615s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:24:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:25:00.458343348 +0000 UTC m=+303.974335832" watchObservedRunningTime="2026-07-15 10:25:00.459034724 +0000 UTC m=+303.975027209" Jul 15 10:25:11.425987 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:11.425944 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-656947bb4-t6ttr" Jul 15 10:25:15.582609 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.582573 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-544bf6b555-cc28c"] Jul 15 10:25:15.593615 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.593584 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:15.595961 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.595932 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 10:25:15.595961 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.595955 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lstmh\"" Jul 15 10:25:15.596166 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.595956 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 10:25:15.605799 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.605774 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-544bf6b555-cc28c"] Jul 15 10:25:15.651906 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.651876 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tls-certs\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:15.652116 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.651924 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6jg5\" (UniqueName: \"kubernetes.io/projected/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-kube-api-access-j6jg5\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:15.652116 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.651977 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tmp\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:15.753041 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.753001 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tls-certs\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:15.753041 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.753045 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6jg5\" (UniqueName: \"kubernetes.io/projected/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-kube-api-access-j6jg5\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:15.753300 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.753078 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tmp\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:15.753300 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:25:15.753181 2585 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 15 10:25:15.753300 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:25:15.753250 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tls-certs podName:4874dd31-0eaa-4dd8-bd1d-12451c4cea88 nodeName:}" failed. No retries permitted until 2026-07-15 10:25:16.253230502 +0000 UTC m=+319.769222968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tls-certs") pod "kube-auth-proxy-544bf6b555-cc28c" (UID: "4874dd31-0eaa-4dd8-bd1d-12451c4cea88") : secret "kube-auth-proxy-tls" not found Jul 15 10:25:15.755449 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.755418 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tmp\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:15.780869 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:15.780837 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6jg5\" (UniqueName: \"kubernetes.io/projected/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-kube-api-access-j6jg5\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:16.257037 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:16.257001 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tls-certs\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:16.259304 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:16.259279 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4874dd31-0eaa-4dd8-bd1d-12451c4cea88-tls-certs\") pod \"kube-auth-proxy-544bf6b555-cc28c\" (UID: \"4874dd31-0eaa-4dd8-bd1d-12451c4cea88\") " pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:16.503501 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:16.503466 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" Jul 15 10:25:16.685278 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:16.685015 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-544bf6b555-cc28c"] Jul 15 10:25:16.692408 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:25:16.692385 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4874dd31_0eaa_4dd8_bd1d_12451c4cea88.slice/crio-a05d9c8e75f5f980ea406b42666bb6481e3df75670b67eac4fd3affc3574fea9 WatchSource:0}: Error finding container a05d9c8e75f5f980ea406b42666bb6481e3df75670b67eac4fd3affc3574fea9: Status 404 returned error can't find the container with id a05d9c8e75f5f980ea406b42666bb6481e3df75670b67eac4fd3affc3574fea9 Jul 15 10:25:17.497947 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.497898 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" event={"ID":"4874dd31-0eaa-4dd8-bd1d-12451c4cea88","Type":"ContainerStarted","Data":"a05d9c8e75f5f980ea406b42666bb6481e3df75670b67eac4fd3affc3574fea9"} Jul 15 10:25:17.516968 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.516940 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5"] Jul 15 10:25:17.519494 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.519474 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.522344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.522143 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 10:25:17.522344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.522179 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f2vt4\"" Jul 15 10:25:17.522684 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.522662 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 10:25:17.522855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.522840 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 10:25:17.523006 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.522992 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 10:25:17.546137 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.545916 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5"] Jul 15 10:25:17.567133 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.567109 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-cacerts\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.567251 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.567155 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-token\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.567251 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.567212 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-local-certs\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.567251 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.567234 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.567447 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.567291 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.567447 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.567349 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhlsf\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-kube-api-access-qhlsf\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.567447 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.567381 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.668239 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.668194 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhlsf\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-kube-api-access-qhlsf\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.668402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.668247 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.668402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.668298 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-cacerts\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.668402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.668333 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-token\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.668402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.668366 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-local-certs\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.668402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.668390 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.668650 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.668438 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.669355 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.669326 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.671606 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.671581 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.671606 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.671602 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-cacerts\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.671808 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.671787 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-local-certs\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.672168 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.672129 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.680652 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.680626 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhlsf\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-kube-api-access-qhlsf\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.680945 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.680925 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-token\") pod \"istiod-openshift-gateway-7574459656-rhqs5\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:17.832975 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:17.832899 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:18.006093 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:18.006049 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5"] Jul 15 10:25:18.009165 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:25:18.009139 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode32fc3fd_dba0_40c3_a9ef_18cc77b8a4df.slice/crio-3a0a6813feeb696cb31ae34afbd52918395855d377c120b8d055739ed828fc5d WatchSource:0}: Error finding container 3a0a6813feeb696cb31ae34afbd52918395855d377c120b8d055739ed828fc5d: Status 404 returned error can't find the container with id 3a0a6813feeb696cb31ae34afbd52918395855d377c120b8d055739ed828fc5d Jul 15 10:25:18.503015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:18.502980 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" event={"ID":"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df","Type":"ContainerStarted","Data":"3a0a6813feeb696cb31ae34afbd52918395855d377c120b8d055739ed828fc5d"} Jul 15 10:25:20.515544 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:20.515504 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" event={"ID":"4874dd31-0eaa-4dd8-bd1d-12451c4cea88","Type":"ContainerStarted","Data":"dde379cecf1e54db3e25750bd977261e04cd360ded3d12ceb9ea962870a37652"} Jul 15 10:25:20.537911 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:20.537858 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-544bf6b555-cc28c" podStartSLOduration=2.360008364 podStartE2EDuration="5.537841701s" totalImagesPullingTime="3.177833337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:25:20.536201597 +0000 UTC m=+324.052194081" watchObservedRunningTime="2026-07-15 10:25:20.537841701 +0000 UTC m=+324.053834184" Jul 15 10:25:20.994675 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:20.994635 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:25:20.994784 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:20.994712 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:25:21.523344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:21.523303 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" event={"ID":"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df","Type":"ContainerStarted","Data":"661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520"} Jul 15 10:25:21.523774 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:21.523493 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:21.552733 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:21.552688 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" podStartSLOduration=1.569863446 podStartE2EDuration="4.552676008s" totalImagesPullingTime="2.982812562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:25:21.55117102 +0000 UTC m=+325.067163504" watchObservedRunningTime="2026-07-15 10:25:21.552676008 +0000 UTC m=+325.068668492" Jul 15 10:25:22.529051 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:22.529026 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:27.102493 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.102462 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6"] Jul 15 10:25:27.105719 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.105700 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:27.110860 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.110662 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 10:25:27.110981 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.110958 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4zcq5\"" Jul 15 10:25:27.125438 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.124093 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m"] Jul 15 10:25:27.129483 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.129427 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:27.132992 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.132837 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7fptv\"" Jul 15 10:25:27.134728 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.134705 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6"] Jul 15 10:25:27.135865 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.135849 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 10:25:27.145901 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.145879 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc6c6de1-4870-4ad0-963c-20fb21390c73-cert\") pod \"kserve-controller-manager-d6f4b65b8-vvpw6\" (UID: \"bc6c6de1-4870-4ad0-963c-20fb21390c73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:27.146083 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.145954 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q548h\" (UniqueName: \"kubernetes.io/projected/bc6c6de1-4870-4ad0-963c-20fb21390c73-kube-api-access-q548h\") pod \"kserve-controller-manager-d6f4b65b8-vvpw6\" (UID: \"bc6c6de1-4870-4ad0-963c-20fb21390c73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:27.147292 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.147274 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m"] Jul 15 10:25:27.246360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.246335 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6tbf\" (UniqueName: \"kubernetes.io/projected/28ab1574-7658-4b02-8fff-8c3d11f2b1d9-kube-api-access-x6tbf\") pod \"llmisvc-controller-manager-777c6bd7d5-95l2m\" (UID: \"28ab1574-7658-4b02-8fff-8c3d11f2b1d9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:27.246467 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.246387 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc6c6de1-4870-4ad0-963c-20fb21390c73-cert\") pod \"kserve-controller-manager-d6f4b65b8-vvpw6\" (UID: \"bc6c6de1-4870-4ad0-963c-20fb21390c73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:27.246467 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.246423 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28ab1574-7658-4b02-8fff-8c3d11f2b1d9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-95l2m\" (UID: \"28ab1574-7658-4b02-8fff-8c3d11f2b1d9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:27.246467 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.246459 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q548h\" (UniqueName: \"kubernetes.io/projected/bc6c6de1-4870-4ad0-963c-20fb21390c73-kube-api-access-q548h\") pod \"kserve-controller-manager-d6f4b65b8-vvpw6\" (UID: \"bc6c6de1-4870-4ad0-963c-20fb21390c73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:27.246623 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:25:27.246550 2585 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 15 10:25:27.246623 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:25:27.246613 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6c6de1-4870-4ad0-963c-20fb21390c73-cert podName:bc6c6de1-4870-4ad0-963c-20fb21390c73 nodeName:}" failed. No retries permitted until 2026-07-15 10:25:27.746592708 +0000 UTC m=+331.262585184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bc6c6de1-4870-4ad0-963c-20fb21390c73-cert") pod "kserve-controller-manager-d6f4b65b8-vvpw6" (UID: "bc6c6de1-4870-4ad0-963c-20fb21390c73") : secret "kserve-webhook-server-cert" not found Jul 15 10:25:27.279472 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.279452 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q548h\" (UniqueName: \"kubernetes.io/projected/bc6c6de1-4870-4ad0-963c-20fb21390c73-kube-api-access-q548h\") pod \"kserve-controller-manager-d6f4b65b8-vvpw6\" (UID: \"bc6c6de1-4870-4ad0-963c-20fb21390c73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:27.347406 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.347383 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6tbf\" (UniqueName: \"kubernetes.io/projected/28ab1574-7658-4b02-8fff-8c3d11f2b1d9-kube-api-access-x6tbf\") pod \"llmisvc-controller-manager-777c6bd7d5-95l2m\" (UID: \"28ab1574-7658-4b02-8fff-8c3d11f2b1d9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:27.347493 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.347449 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28ab1574-7658-4b02-8fff-8c3d11f2b1d9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-95l2m\" (UID: \"28ab1574-7658-4b02-8fff-8c3d11f2b1d9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:27.349631 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.349614 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28ab1574-7658-4b02-8fff-8c3d11f2b1d9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-95l2m\" (UID: \"28ab1574-7658-4b02-8fff-8c3d11f2b1d9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:27.360245 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.360190 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6tbf\" (UniqueName: \"kubernetes.io/projected/28ab1574-7658-4b02-8fff-8c3d11f2b1d9-kube-api-access-x6tbf\") pod \"llmisvc-controller-manager-777c6bd7d5-95l2m\" (UID: \"28ab1574-7658-4b02-8fff-8c3d11f2b1d9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:27.439769 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.439749 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:27.571128 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.571092 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m"] Jul 15 10:25:27.577134 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:25:27.577101 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod28ab1574_7658_4b02_8fff_8c3d11f2b1d9.slice/crio-d0428c21cae4b1b6cc0ff305468ad4ec96efa77eaebf27f58e1377d422c905eb WatchSource:0}: Error finding container d0428c21cae4b1b6cc0ff305468ad4ec96efa77eaebf27f58e1377d422c905eb: Status 404 returned error can't find the container with id d0428c21cae4b1b6cc0ff305468ad4ec96efa77eaebf27f58e1377d422c905eb Jul 15 10:25:27.750821 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.750787 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc6c6de1-4870-4ad0-963c-20fb21390c73-cert\") pod \"kserve-controller-manager-d6f4b65b8-vvpw6\" (UID: \"bc6c6de1-4870-4ad0-963c-20fb21390c73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:27.753074 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:27.753030 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc6c6de1-4870-4ad0-963c-20fb21390c73-cert\") pod \"kserve-controller-manager-d6f4b65b8-vvpw6\" (UID: \"bc6c6de1-4870-4ad0-963c-20fb21390c73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:28.016295 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:28.016225 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:28.146934 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:28.146908 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6"] Jul 15 10:25:28.148570 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:25:28.148547 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6c6de1_4870_4ad0_963c_20fb21390c73.slice/crio-d65e1a0886e03945cfbb6d322639bffe36dda6f8b27b6a3b124a3912fa2df063 WatchSource:0}: Error finding container d65e1a0886e03945cfbb6d322639bffe36dda6f8b27b6a3b124a3912fa2df063: Status 404 returned error can't find the container with id d65e1a0886e03945cfbb6d322639bffe36dda6f8b27b6a3b124a3912fa2df063 Jul 15 10:25:28.556271 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:28.556231 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" event={"ID":"bc6c6de1-4870-4ad0-963c-20fb21390c73","Type":"ContainerStarted","Data":"d65e1a0886e03945cfbb6d322639bffe36dda6f8b27b6a3b124a3912fa2df063"} Jul 15 10:25:28.557699 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:28.557673 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" event={"ID":"28ab1574-7658-4b02-8fff-8c3d11f2b1d9","Type":"ContainerStarted","Data":"d0428c21cae4b1b6cc0ff305468ad4ec96efa77eaebf27f58e1377d422c905eb"} Jul 15 10:25:29.273738 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.273698 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5"] Jul 15 10:25:29.274251 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.273908 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" podUID="e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" containerName="discovery" containerID="cri-o://661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520" gracePeriod=30 Jul 15 10:25:29.546966 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.546943 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:29.563243 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.563218 2585 generic.go:356] "Generic (PLEG): container finished" podID="e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" containerID="661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520" exitCode=0 Jul 15 10:25:29.563360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.563262 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" Jul 15 10:25:29.563360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.563272 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" event={"ID":"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df","Type":"ContainerDied","Data":"661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520"} Jul 15 10:25:29.563360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.563311 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5" event={"ID":"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df","Type":"ContainerDied","Data":"3a0a6813feeb696cb31ae34afbd52918395855d377c120b8d055739ed828fc5d"} Jul 15 10:25:29.563360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.563335 2585 scope.go:117] "RemoveContainer" containerID="661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520" Jul 15 10:25:29.570990 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.570970 2585 scope.go:117] "RemoveContainer" containerID="661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520" Jul 15 10:25:29.571243 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:25:29.571226 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520\": container with ID starting with 661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520 not found: ID does not exist" containerID="661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520" Jul 15 10:25:29.571301 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.571255 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520"} err="failed to get container status \"661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520\": rpc error: code = NotFound desc = could not find container \"661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520\": container with ID starting with 661be396265d9345d4d4f1f9b94e4330c67aaaa5bde6a44e42538b84acf63520 not found: ID does not exist" Jul 15 10:25:29.666906 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.666881 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-token\") pod \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " Jul 15 10:25:29.667040 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.666936 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-local-certs\") pod \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " Jul 15 10:25:29.667040 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.666960 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-dns-cert\") pod \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " Jul 15 10:25:29.667040 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.666985 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhlsf\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-kube-api-access-qhlsf\") pod \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " Jul 15 10:25:29.667185 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.667041 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-cacerts\") pod \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " Jul 15 10:25:29.667185 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.667095 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-ca-configmap\") pod \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " Jul 15 10:25:29.667185 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.667124 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-kubeconfig\") pod \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\" (UID: \"e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df\") " Jul 15 10:25:29.667545 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.667509 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" (UID: "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:25:29.669355 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.669328 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-local-certs" (OuterVolumeSpecName: "local-certs") pod "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" (UID: "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:25:29.669803 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.669776 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-kube-api-access-qhlsf" (OuterVolumeSpecName: "kube-api-access-qhlsf") pod "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" (UID: "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df"). InnerVolumeSpecName "kube-api-access-qhlsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:25:29.669899 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.669785 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-token" (OuterVolumeSpecName: "istio-token") pod "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" (UID: "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:25:29.669899 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.669870 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" (UID: "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:25:29.669991 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.669897 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" (UID: "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:25:29.669991 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.669925 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-cacerts" (OuterVolumeSpecName: "cacerts") pod "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" (UID: "e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:25:29.768126 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.768102 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:25:29.768384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.768129 2585 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-local-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:25:29.768384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.768144 2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-dns-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:25:29.768384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.768156 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhlsf\" (UniqueName: \"kubernetes.io/projected/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-kube-api-access-qhlsf\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:25:29.768384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.768170 2585 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-cacerts\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:25:29.768384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.768183 2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-csr-ca-configmap\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:25:29.768384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.768195 2585 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df-istio-kubeconfig\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:25:29.891364 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.891334 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5"] Jul 15 10:25:29.895051 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:29.895026 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rhqs5"] Jul 15 10:25:30.934478 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:30.934441 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" path="/var/lib/kubelet/pods/e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df/volumes" Jul 15 10:25:31.575658 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:31.575621 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" event={"ID":"28ab1574-7658-4b02-8fff-8c3d11f2b1d9","Type":"ContainerStarted","Data":"5b0fb4e3e560105482cbd2cf0e6a19e7a499664acad835bc7423f961749e908f"} Jul 15 10:25:31.575860 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:31.575713 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:25:31.577167 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:31.577145 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" event={"ID":"bc6c6de1-4870-4ad0-963c-20fb21390c73","Type":"ContainerStarted","Data":"7017531c339403d5b9c85751b8a62476cb04f0dda71ea423d616e34ccb97fbb3"} Jul 15 10:25:31.577274 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:31.577186 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:25:31.632180 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:25:31.632138 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" podStartSLOduration=1.07959669 podStartE2EDuration="4.632124716s" totalImagesPullingTime="3.552528026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:25:31.608746974 +0000 UTC m=+335.124739457" watchObservedRunningTime="2026-07-15 10:25:31.632124716 +0000 UTC m=+335.148117200" Jul 15 10:26:02.583705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:02.583673 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95l2m" Jul 15 10:26:02.586843 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:02.586825 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" Jul 15 10:26:02.603156 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:02.603113 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vvpw6" podStartSLOduration=32.625510188 podStartE2EDuration="35.603097948s" totalImagesPullingTime="2.97758776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:25:31.631823848 +0000 UTC m=+335.147816366" watchObservedRunningTime="2026-07-15 10:26:02.603097948 +0000 UTC m=+366.119090434" Jul 15 10:26:16.529237 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.529205 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tchn"] Jul 15 10:26:16.529671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.529534 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" containerName="discovery" Jul 15 10:26:16.529671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.529544 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" containerName="discovery" Jul 15 10:26:16.529671 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.529603 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e32fc3fd-dba0-40c3-a9ef-18cc77b8a4df" containerName="discovery" Jul 15 10:26:16.537426 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.537409 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" Jul 15 10:26:16.540716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.540697 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lfjr9\"" Jul 15 10:26:16.541689 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.541672 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 10:26:16.541753 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.541672 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 10:26:16.545976 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.545957 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tchn"] Jul 15 10:26:16.714753 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.714727 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjgwl\" (UniqueName: \"kubernetes.io/projected/28f58d11-ce89-40a9-9731-a396e5243a7f-kube-api-access-hjgwl\") pod \"kuadrant-operator-catalog-9tchn\" (UID: \"28f58d11-ce89-40a9-9731-a396e5243a7f\") " pod="kuadrant-system/kuadrant-operator-catalog-9tchn" Jul 15 10:26:16.815835 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.815771 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjgwl\" (UniqueName: \"kubernetes.io/projected/28f58d11-ce89-40a9-9731-a396e5243a7f-kube-api-access-hjgwl\") pod \"kuadrant-operator-catalog-9tchn\" (UID: \"28f58d11-ce89-40a9-9731-a396e5243a7f\") " pod="kuadrant-system/kuadrant-operator-catalog-9tchn" Jul 15 10:26:16.825541 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.825519 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjgwl\" (UniqueName: \"kubernetes.io/projected/28f58d11-ce89-40a9-9731-a396e5243a7f-kube-api-access-hjgwl\") pod \"kuadrant-operator-catalog-9tchn\" (UID: \"28f58d11-ce89-40a9-9731-a396e5243a7f\") " pod="kuadrant-system/kuadrant-operator-catalog-9tchn" Jul 15 10:26:16.846979 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.846958 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" Jul 15 10:26:16.980828 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:16.980803 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tchn"] Jul 15 10:26:16.985461 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:26:16.985434 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f58d11_ce89_40a9_9731_a396e5243a7f.slice/crio-386fb585ed7deef4c1c734a2e4c80441b2cb875e986dfc9cdd6e8525577a5dc5 WatchSource:0}: Error finding container 386fb585ed7deef4c1c734a2e4c80441b2cb875e986dfc9cdd6e8525577a5dc5: Status 404 returned error can't find the container with id 386fb585ed7deef4c1c734a2e4c80441b2cb875e986dfc9cdd6e8525577a5dc5 Jul 15 10:26:17.043895 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.043866 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tchn"] Jul 15 10:26:17.080163 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.080112 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g6n4q"] Jul 15 10:26:17.082741 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.082725 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:17.099845 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.099824 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g6n4q"] Jul 15 10:26:17.219444 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.219419 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q62vh\" (UniqueName: \"kubernetes.io/projected/a91b7f6c-79b3-41e9-b724-9155a0ad9780-kube-api-access-q62vh\") pod \"kuadrant-operator-catalog-g6n4q\" (UID: \"a91b7f6c-79b3-41e9-b724-9155a0ad9780\") " pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:17.319800 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.319779 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q62vh\" (UniqueName: \"kubernetes.io/projected/a91b7f6c-79b3-41e9-b724-9155a0ad9780-kube-api-access-q62vh\") pod \"kuadrant-operator-catalog-g6n4q\" (UID: \"a91b7f6c-79b3-41e9-b724-9155a0ad9780\") " pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:17.330034 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.330014 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q62vh\" (UniqueName: \"kubernetes.io/projected/a91b7f6c-79b3-41e9-b724-9155a0ad9780-kube-api-access-q62vh\") pod \"kuadrant-operator-catalog-g6n4q\" (UID: \"a91b7f6c-79b3-41e9-b724-9155a0ad9780\") " pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:17.391705 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.391648 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:17.561980 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.561958 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g6n4q"] Jul 15 10:26:17.564413 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:26:17.564384 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91b7f6c_79b3_41e9_b724_9155a0ad9780.slice/crio-e2694ae3d3677fb64353638f8990585161a082cfc1b46c3cf6b3b1d69231c790 WatchSource:0}: Error finding container e2694ae3d3677fb64353638f8990585161a082cfc1b46c3cf6b3b1d69231c790: Status 404 returned error can't find the container with id e2694ae3d3677fb64353638f8990585161a082cfc1b46c3cf6b3b1d69231c790 Jul 15 10:26:17.786717 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.786685 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" event={"ID":"28f58d11-ce89-40a9-9731-a396e5243a7f","Type":"ContainerStarted","Data":"386fb585ed7deef4c1c734a2e4c80441b2cb875e986dfc9cdd6e8525577a5dc5"} Jul 15 10:26:17.787894 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:17.787866 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" event={"ID":"a91b7f6c-79b3-41e9-b724-9155a0ad9780","Type":"ContainerStarted","Data":"e2694ae3d3677fb64353638f8990585161a082cfc1b46c3cf6b3b1d69231c790"} Jul 15 10:26:19.797911 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:19.797873 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" event={"ID":"28f58d11-ce89-40a9-9731-a396e5243a7f","Type":"ContainerStarted","Data":"7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6"} Jul 15 10:26:19.798367 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:19.797932 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" podUID="28f58d11-ce89-40a9-9731-a396e5243a7f" containerName="registry-server" containerID="cri-o://7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6" gracePeriod=2 Jul 15 10:26:19.799453 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:19.799426 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" event={"ID":"a91b7f6c-79b3-41e9-b724-9155a0ad9780","Type":"ContainerStarted","Data":"b4e3982f3413ed997bdd711b841e2423fa2929dbfa7c06d58a3d8fc64d8896db"} Jul 15 10:26:19.816943 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:19.816908 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" podStartSLOduration=1.891614739 podStartE2EDuration="3.816897422s" totalImagesPullingTime="1.925282683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:26:19.815885342 +0000 UTC m=+383.331877827" watchObservedRunningTime="2026-07-15 10:26:19.816897422 +0000 UTC m=+383.332889906" Jul 15 10:26:19.834233 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:19.834195 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" podStartSLOduration=1.487900116 podStartE2EDuration="2.834183678s" totalImagesPullingTime="1.346283562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:26:19.833088448 +0000 UTC m=+383.349080925" watchObservedRunningTime="2026-07-15 10:26:19.834183678 +0000 UTC m=+383.350176163" Jul 15 10:26:20.036542 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.036519 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" Jul 15 10:26:20.044207 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.044188 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjgwl\" (UniqueName: \"kubernetes.io/projected/28f58d11-ce89-40a9-9731-a396e5243a7f-kube-api-access-hjgwl\") pod \"28f58d11-ce89-40a9-9731-a396e5243a7f\" (UID: \"28f58d11-ce89-40a9-9731-a396e5243a7f\") " Jul 15 10:26:20.046092 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.046056 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f58d11-ce89-40a9-9731-a396e5243a7f-kube-api-access-hjgwl" (OuterVolumeSpecName: "kube-api-access-hjgwl") pod "28f58d11-ce89-40a9-9731-a396e5243a7f" (UID: "28f58d11-ce89-40a9-9731-a396e5243a7f"). InnerVolumeSpecName "kube-api-access-hjgwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:26:20.145709 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.145686 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjgwl\" (UniqueName: \"kubernetes.io/projected/28f58d11-ce89-40a9-9731-a396e5243a7f-kube-api-access-hjgwl\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:26:20.805302 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.805275 2585 generic.go:356] "Generic (PLEG): container finished" podID="28f58d11-ce89-40a9-9731-a396e5243a7f" containerID="7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6" exitCode=0 Jul 15 10:26:20.805658 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.805319 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" Jul 15 10:26:20.805658 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.805361 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" event={"ID":"28f58d11-ce89-40a9-9731-a396e5243a7f","Type":"ContainerDied","Data":"7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6"} Jul 15 10:26:20.805658 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.805425 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9tchn" event={"ID":"28f58d11-ce89-40a9-9731-a396e5243a7f","Type":"ContainerDied","Data":"386fb585ed7deef4c1c734a2e4c80441b2cb875e986dfc9cdd6e8525577a5dc5"} Jul 15 10:26:20.805658 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.805447 2585 scope.go:117] "RemoveContainer" containerID="7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6" Jul 15 10:26:20.814481 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.814466 2585 scope.go:117] "RemoveContainer" containerID="7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6" Jul 15 10:26:20.814761 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:26:20.814734 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6\": container with ID starting with 7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6 not found: ID does not exist" containerID="7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6" Jul 15 10:26:20.814843 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.814768 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6"} err="failed to get container status \"7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6\": rpc error: code = NotFound desc = could not find container \"7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6\": container with ID starting with 7703cd1e772c3bdd7bc96602ae56c096f6ae3fcc56021f000088ff57aa9b44c6 not found: ID does not exist" Jul 15 10:26:20.829833 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.829813 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tchn"] Jul 15 10:26:20.836046 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.836010 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tchn"] Jul 15 10:26:20.932558 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:20.932538 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28f58d11-ce89-40a9-9731-a396e5243a7f" path="/var/lib/kubelet/pods/28f58d11-ce89-40a9-9731-a396e5243a7f/volumes" Jul 15 10:26:27.392003 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:27.391961 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:27.392582 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:27.392035 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:27.415588 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:27.415561 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:27.859852 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:27.859826 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-g6n4q" Jul 15 10:26:35.782783 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.782752 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78945ccfbf-pgd25"] Jul 15 10:26:35.783139 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.783100 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f58d11-ce89-40a9-9731-a396e5243a7f" containerName="registry-server" Jul 15 10:26:35.783139 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.783113 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f58d11-ce89-40a9-9731-a396e5243a7f" containerName="registry-server" Jul 15 10:26:35.783240 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.783170 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f58d11-ce89-40a9-9731-a396e5243a7f" containerName="registry-server" Jul 15 10:26:35.787327 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.787304 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.790937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.790762 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6zk45\"" Jul 15 10:26:35.790937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.790779 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 10:26:35.790937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.790817 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 10:26:35.790937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.790828 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 10:26:35.790937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.790849 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 10:26:35.790937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.790862 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 10:26:35.791366 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.791347 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 10:26:35.791495 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.791370 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 10:26:35.794557 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.794464 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 10:26:35.804551 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.804533 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78945ccfbf-pgd25"] Jul 15 10:26:35.857974 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.857952 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-service-ca\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.858108 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.857993 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-serving-cert\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.858108 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.858022 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-trusted-ca-bundle\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.858203 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.858102 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rlqq\" (UniqueName: \"kubernetes.io/projected/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-kube-api-access-8rlqq\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.858203 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.858127 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-oauth-config\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.858203 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.858150 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-oauth-serving-cert\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.858203 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.858169 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-config\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.959269 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.959247 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-serving-cert\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.959384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.959277 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-trusted-ca-bundle\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.959384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.959301 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rlqq\" (UniqueName: \"kubernetes.io/projected/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-kube-api-access-8rlqq\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.959384 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.959318 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-oauth-config\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.959539 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.959454 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-oauth-serving-cert\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.959539 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.959499 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-config\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.959643 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.959555 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-service-ca\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.960189 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.960167 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-oauth-serving-cert\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.960290 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.960216 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-trusted-ca-bundle\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.960290 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.960278 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-service-ca\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.960368 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.960336 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-config\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.961811 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.961786 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-oauth-config\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.961892 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.961846 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-console-serving-cert\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:35.972257 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:35.972233 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rlqq\" (UniqueName: \"kubernetes.io/projected/ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea-kube-api-access-8rlqq\") pod \"console-78945ccfbf-pgd25\" (UID: \"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea\") " pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:36.096654 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:36.096602 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:36.241311 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:36.241287 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78945ccfbf-pgd25"] Jul 15 10:26:36.243488 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:26:36.243460 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4a8c65_4ed4_4de5_8f3b_2f8877d285ea.slice/crio-58cc2c3da50e2e2538a5bbbbd6f1282437acc85ff163336eb02b67b0c3d2a128 WatchSource:0}: Error finding container 58cc2c3da50e2e2538a5bbbbd6f1282437acc85ff163336eb02b67b0c3d2a128: Status 404 returned error can't find the container with id 58cc2c3da50e2e2538a5bbbbd6f1282437acc85ff163336eb02b67b0c3d2a128 Jul 15 10:26:36.880527 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:36.880487 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78945ccfbf-pgd25" event={"ID":"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea","Type":"ContainerStarted","Data":"7739be8069c6c009fc3c0709971c3d0208d2f17ed63899aed9b52edd9e39e6e2"} Jul 15 10:26:36.880877 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:36.880578 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78945ccfbf-pgd25" event={"ID":"ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea","Type":"ContainerStarted","Data":"58cc2c3da50e2e2538a5bbbbd6f1282437acc85ff163336eb02b67b0c3d2a128"} Jul 15 10:26:36.901575 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:36.901534 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78945ccfbf-pgd25" podStartSLOduration=1.901519361 podStartE2EDuration="1.901519361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:26:36.90066718 +0000 UTC m=+400.416659665" watchObservedRunningTime="2026-07-15 10:26:36.901519361 +0000 UTC m=+400.417511848" Jul 15 10:26:46.097323 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:46.097275 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:46.097323 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:46.097336 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:46.102264 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:46.102234 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:46.933500 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:46.933474 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78945ccfbf-pgd25" Jul 15 10:26:48.181415 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.181384 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g"] Jul 15 10:26:48.187493 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.187471 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:26:48.190039 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.190020 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-p6nhj\"" Jul 15 10:26:48.197199 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.197179 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g"] Jul 15 10:26:48.252307 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.252275 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnqvn\" (UniqueName: \"kubernetes.io/projected/2084243c-3b01-4d8a-a47e-000083f65f0b-kube-api-access-xnqvn\") pod \"limitador-operator-controller-manager-859d7b78bd-6ds4g\" (UID: \"2084243c-3b01-4d8a-a47e-000083f65f0b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:26:48.353328 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.353300 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnqvn\" (UniqueName: \"kubernetes.io/projected/2084243c-3b01-4d8a-a47e-000083f65f0b-kube-api-access-xnqvn\") pod \"limitador-operator-controller-manager-859d7b78bd-6ds4g\" (UID: \"2084243c-3b01-4d8a-a47e-000083f65f0b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:26:48.366456 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.366435 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnqvn\" (UniqueName: \"kubernetes.io/projected/2084243c-3b01-4d8a-a47e-000083f65f0b-kube-api-access-xnqvn\") pod \"limitador-operator-controller-manager-859d7b78bd-6ds4g\" (UID: \"2084243c-3b01-4d8a-a47e-000083f65f0b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:26:48.498057 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.498036 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:26:48.628599 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.628534 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g"] Jul 15 10:26:48.631502 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:26:48.631478 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2084243c_3b01_4d8a_a47e_000083f65f0b.slice/crio-3aeec344be4443de855da8a9c2985e32a132558dda347e171b4a8046317444af WatchSource:0}: Error finding container 3aeec344be4443de855da8a9c2985e32a132558dda347e171b4a8046317444af: Status 404 returned error can't find the container with id 3aeec344be4443de855da8a9c2985e32a132558dda347e171b4a8046317444af Jul 15 10:26:48.935816 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:48.935754 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" event={"ID":"2084243c-3b01-4d8a-a47e-000083f65f0b","Type":"ContainerStarted","Data":"3aeec344be4443de855da8a9c2985e32a132558dda347e171b4a8046317444af"} Jul 15 10:26:50.946098 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:50.945988 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" event={"ID":"2084243c-3b01-4d8a-a47e-000083f65f0b","Type":"ContainerStarted","Data":"db275146878fe6f5b49a9c88abf9d95db1a3a1d147005d76b18a8ee3b0d6d292"} Jul 15 10:26:50.946098 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:50.946053 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:26:50.966424 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:50.966375 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" podStartSLOduration=1.042292578 podStartE2EDuration="2.966362487s" totalImagesPullingTime="1.924069909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:26:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:26:50.964834254 +0000 UTC m=+414.480826740" watchObservedRunningTime="2026-07-15 10:26:50.966362487 +0000 UTC m=+414.482354972" Jul 15 10:26:51.525941 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.525906 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r"] Jul 15 10:26:51.529541 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.529516 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" Jul 15 10:26:51.532002 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.531985 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 10:26:51.532002 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.531996 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-cjdzk\"" Jul 15 10:26:51.543050 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.543027 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r"] Jul 15 10:26:51.579024 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.578996 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gln65\" (UniqueName: \"kubernetes.io/projected/1fe14792-29de-4773-866c-da83239e45ef-kube-api-access-gln65\") pod \"dns-operator-controller-manager-864dd78c4c-kjd4r\" (UID: \"1fe14792-29de-4773-866c-da83239e45ef\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" Jul 15 10:26:51.679859 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.679837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gln65\" (UniqueName: \"kubernetes.io/projected/1fe14792-29de-4773-866c-da83239e45ef-kube-api-access-gln65\") pod \"dns-operator-controller-manager-864dd78c4c-kjd4r\" (UID: \"1fe14792-29de-4773-866c-da83239e45ef\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" Jul 15 10:26:51.688984 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.688962 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gln65\" (UniqueName: \"kubernetes.io/projected/1fe14792-29de-4773-866c-da83239e45ef-kube-api-access-gln65\") pod \"dns-operator-controller-manager-864dd78c4c-kjd4r\" (UID: \"1fe14792-29de-4773-866c-da83239e45ef\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" Jul 15 10:26:51.839356 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.839304 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" Jul 15 10:26:51.965011 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:51.964987 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r"] Jul 15 10:26:51.966739 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:26:51.966706 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe14792_29de_4773_866c_da83239e45ef.slice/crio-1f7e47e6ddb1b99e2d81f7d0b59c265e673068287bc9fd9daffd82e8c0f6eb2e WatchSource:0}: Error finding container 1f7e47e6ddb1b99e2d81f7d0b59c265e673068287bc9fd9daffd82e8c0f6eb2e: Status 404 returned error can't find the container with id 1f7e47e6ddb1b99e2d81f7d0b59c265e673068287bc9fd9daffd82e8c0f6eb2e Jul 15 10:26:52.959796 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:52.959760 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" event={"ID":"1fe14792-29de-4773-866c-da83239e45ef","Type":"ContainerStarted","Data":"1f7e47e6ddb1b99e2d81f7d0b59c265e673068287bc9fd9daffd82e8c0f6eb2e"} Jul 15 10:26:53.966733 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:53.966708 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" event={"ID":"1fe14792-29de-4773-866c-da83239e45ef","Type":"ContainerStarted","Data":"afec0feb2885c11df9be817c32eff7a46e6ed27648a202c19338a61fb9773a7d"} Jul 15 10:26:53.967057 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:53.966837 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" Jul 15 10:26:53.986073 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:53.986019 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" podStartSLOduration=1.06279248 podStartE2EDuration="2.986008402s" totalImagesPullingTime="1.923215922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:26:53.985030718 +0000 UTC m=+417.501023207" watchObservedRunningTime="2026-07-15 10:26:53.986008402 +0000 UTC m=+417.502000887" Jul 15 10:26:56.505089 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.505038 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2"] Jul 15 10:26:56.508425 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.508407 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:26:56.511475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.511444 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gjg8k\"" Jul 15 10:26:56.522950 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.522925 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2"] Jul 15 10:26:56.618124 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.618101 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-645wb\" (UniqueName: \"kubernetes.io/projected/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-kube-api-access-645wb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" (UID: \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:26:56.618233 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.618129 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" (UID: \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:26:56.719431 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.719409 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-645wb\" (UniqueName: \"kubernetes.io/projected/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-kube-api-access-645wb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" (UID: \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:26:56.719548 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.719444 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" (UID: \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:26:56.719778 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.719758 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" (UID: \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:26:56.729317 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.729295 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-645wb\" (UniqueName: \"kubernetes.io/projected/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-kube-api-access-645wb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" (UID: \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:26:56.821033 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.820982 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gjg8k\"" Jul 15 10:26:56.828942 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.828923 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:26:56.973908 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:56.973889 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2"] Jul 15 10:26:56.976990 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:26:56.976965 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa37c34_b5cb_445a_a111_5d0e9838f8b5.slice/crio-4f8f82d470ebf1554f64be73a115f34d77db3057debb1cd09b231572ca137040 WatchSource:0}: Error finding container 4f8f82d470ebf1554f64be73a115f34d77db3057debb1cd09b231572ca137040: Status 404 returned error can't find the container with id 4f8f82d470ebf1554f64be73a115f34d77db3057debb1cd09b231572ca137040 Jul 15 10:26:57.985768 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:26:57.985735 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" event={"ID":"dfa37c34-b5cb-445a-a111-5d0e9838f8b5","Type":"ContainerStarted","Data":"4f8f82d470ebf1554f64be73a115f34d77db3057debb1cd09b231572ca137040"} Jul 15 10:27:00.334910 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.334876 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lt79z"] Jul 15 10:27:00.338253 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.338237 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" Jul 15 10:27:00.340598 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.340577 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s9c7l\"" Jul 15 10:27:00.354897 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.354876 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lt79z"] Jul 15 10:27:00.450333 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.450308 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72vvc\" (UniqueName: \"kubernetes.io/projected/3490c1cd-5a43-4ac9-a4e2-27a75f05874d-kube-api-access-72vvc\") pod \"authorino-operator-75cc8d5cb7-lt79z\" (UID: \"3490c1cd-5a43-4ac9-a4e2-27a75f05874d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" Jul 15 10:27:00.551048 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.551019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72vvc\" (UniqueName: \"kubernetes.io/projected/3490c1cd-5a43-4ac9-a4e2-27a75f05874d-kube-api-access-72vvc\") pod \"authorino-operator-75cc8d5cb7-lt79z\" (UID: \"3490c1cd-5a43-4ac9-a4e2-27a75f05874d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" Jul 15 10:27:00.560645 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.560620 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72vvc\" (UniqueName: \"kubernetes.io/projected/3490c1cd-5a43-4ac9-a4e2-27a75f05874d-kube-api-access-72vvc\") pod \"authorino-operator-75cc8d5cb7-lt79z\" (UID: \"3490c1cd-5a43-4ac9-a4e2-27a75f05874d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" Jul 15 10:27:00.647910 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.647857 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" Jul 15 10:27:00.773427 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:00.773269 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lt79z"] Jul 15 10:27:00.775644 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:27:00.775616 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3490c1cd_5a43_4ac9_a4e2_27a75f05874d.slice/crio-13b91cae3d240a92e876323bbb9e74f13d9fa9bb80961703bf7fc4ad8b981b3d WatchSource:0}: Error finding container 13b91cae3d240a92e876323bbb9e74f13d9fa9bb80961703bf7fc4ad8b981b3d: Status 404 returned error can't find the container with id 13b91cae3d240a92e876323bbb9e74f13d9fa9bb80961703bf7fc4ad8b981b3d Jul 15 10:27:01.000773 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:01.000739 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" event={"ID":"3490c1cd-5a43-4ac9-a4e2-27a75f05874d","Type":"ContainerStarted","Data":"13b91cae3d240a92e876323bbb9e74f13d9fa9bb80961703bf7fc4ad8b981b3d"} Jul 15 10:27:01.954313 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:01.954282 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:27:04.973420 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:04.973390 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kjd4r" Jul 15 10:27:05.023293 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:05.023260 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" event={"ID":"3490c1cd-5a43-4ac9-a4e2-27a75f05874d","Type":"ContainerStarted","Data":"6a8fffe3be0d0dd759c1ec4e1ee627101a52fb6ba6b1f7d3128ebc7fc01d3b68"} Jul 15 10:27:05.023477 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:05.023381 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" Jul 15 10:27:05.025032 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:05.025010 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" event={"ID":"dfa37c34-b5cb-445a-a111-5d0e9838f8b5","Type":"ContainerStarted","Data":"5bee5062e241022b198766ca6bb5d12f32f093292f56dc06810d08a43c8adf57"} Jul 15 10:27:05.025173 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:05.025055 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:05.042691 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:05.042654 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" podStartSLOduration=1.653493906 podStartE2EDuration="5.042642428s" totalImagesPullingTime="3.389148522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:27:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:27:05.041416497 +0000 UTC m=+428.557408983" watchObservedRunningTime="2026-07-15 10:27:05.042642428 +0000 UTC m=+428.558634912" Jul 15 10:27:05.064696 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:05.064648 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" podStartSLOduration=1.8670103249999999 podStartE2EDuration="9.06463964s" totalImagesPullingTime="7.197629315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:27:05.063976677 +0000 UTC m=+428.579969159" watchObservedRunningTime="2026-07-15 10:27:05.06463964 +0000 UTC m=+428.580632124" Jul 15 10:27:06.966197 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:06.966167 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc"] Jul 15 10:27:06.969864 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:06.969843 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:06.972450 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:06.972429 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 15 10:27:06.972450 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:06.972450 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 15 10:27:06.972706 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:06.972690 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-z7ndj\"" Jul 15 10:27:06.978976 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:06.978956 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc"] Jul 15 10:27:07.103043 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.103016 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9g9q\" (UniqueName: \"kubernetes.io/projected/30ad177f-7132-4eae-bdc3-69dd28acf490-kube-api-access-v9g9q\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.103187 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.103093 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30ad177f-7132-4eae-bdc3-69dd28acf490-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.103187 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.103150 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/30ad177f-7132-4eae-bdc3-69dd28acf490-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.204015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.203988 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/30ad177f-7132-4eae-bdc3-69dd28acf490-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.204160 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.204037 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9g9q\" (UniqueName: \"kubernetes.io/projected/30ad177f-7132-4eae-bdc3-69dd28acf490-kube-api-access-v9g9q\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.204160 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.204098 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30ad177f-7132-4eae-bdc3-69dd28acf490-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.204684 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.204668 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30ad177f-7132-4eae-bdc3-69dd28acf490-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.206323 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.206306 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/30ad177f-7132-4eae-bdc3-69dd28acf490-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.213349 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.213321 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9g9q\" (UniqueName: \"kubernetes.io/projected/30ad177f-7132-4eae-bdc3-69dd28acf490-kube-api-access-v9g9q\") pod \"kuadrant-console-plugin-66b5774985-44lpc\" (UID: \"30ad177f-7132-4eae-bdc3-69dd28acf490\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.288215 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.288195 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" Jul 15 10:27:07.415464 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:07.415437 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc"] Jul 15 10:27:07.417548 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:27:07.417525 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30ad177f_7132_4eae_bdc3_69dd28acf490.slice/crio-94f999dad61f2ba4940b2065816fb8e613c980362b48b7f231fe35ce6f402c69 WatchSource:0}: Error finding container 94f999dad61f2ba4940b2065816fb8e613c980362b48b7f231fe35ce6f402c69: Status 404 returned error can't find the container with id 94f999dad61f2ba4940b2065816fb8e613c980362b48b7f231fe35ce6f402c69 Jul 15 10:27:08.038672 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:08.038641 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" event={"ID":"30ad177f-7132-4eae-bdc3-69dd28acf490","Type":"ContainerStarted","Data":"94f999dad61f2ba4940b2065816fb8e613c980362b48b7f231fe35ce6f402c69"} Jul 15 10:27:16.032315 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:16.032280 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:16.032731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:16.032358 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lt79z" Jul 15 10:27:17.402893 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.402858 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2"] Jul 15 10:27:17.424348 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.424307 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:17.425886 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.425862 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2"] Jul 15 10:27:17.599779 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.599743 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-998vm\" (UniqueName: \"kubernetes.io/projected/af7c6127-d72b-438b-bd87-efe2e08f3811-kube-api-access-998vm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" (UID: \"af7c6127-d72b-438b-bd87-efe2e08f3811\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:17.599962 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.599817 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af7c6127-d72b-438b-bd87-efe2e08f3811-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" (UID: \"af7c6127-d72b-438b-bd87-efe2e08f3811\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:17.700457 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.700370 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af7c6127-d72b-438b-bd87-efe2e08f3811-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" (UID: \"af7c6127-d72b-438b-bd87-efe2e08f3811\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:17.700613 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.700503 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-998vm\" (UniqueName: \"kubernetes.io/projected/af7c6127-d72b-438b-bd87-efe2e08f3811-kube-api-access-998vm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" (UID: \"af7c6127-d72b-438b-bd87-efe2e08f3811\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:17.700787 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.700768 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af7c6127-d72b-438b-bd87-efe2e08f3811-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" (UID: \"af7c6127-d72b-438b-bd87-efe2e08f3811\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:17.720163 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.720135 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-998vm\" (UniqueName: \"kubernetes.io/projected/af7c6127-d72b-438b-bd87-efe2e08f3811-kube-api-access-998vm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" (UID: \"af7c6127-d72b-438b-bd87-efe2e08f3811\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:17.738731 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:17.738704 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:18.065824 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.065789 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2"] Jul 15 10:27:18.093889 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.093854 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2"] Jul 15 10:27:18.106463 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.106428 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv"] Jul 15 10:27:18.119099 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.118826 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2"] Jul 15 10:27:18.119099 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.118909 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:18.119297 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.119048 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" containerName="manager" containerID="cri-o://5bee5062e241022b198766ca6bb5d12f32f093292f56dc06810d08a43c8adf57" gracePeriod=2 Jul 15 10:27:18.124054 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.122332 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2"] Jul 15 10:27:18.124054 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:18.122801 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:18.127315 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.127257 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv"] Jul 15 10:27:18.140747 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.140671 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g"] Jul 15 10:27:18.140976 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.140878 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" containerName="manager" containerID="cri-o://db275146878fe6f5b49a9c88abf9d95db1a3a1d147005d76b18a8ee3b0d6d292" gracePeriod=2 Jul 15 10:27:18.146416 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.146395 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6"] Jul 15 10:27:18.147003 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.146983 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" containerName="manager" Jul 15 10:27:18.147374 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.147006 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" containerName="manager" Jul 15 10:27:18.147374 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.147150 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" containerName="manager" Jul 15 10:27:18.151193 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.151160 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:18.154049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.154028 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g"] Jul 15 10:27:18.160634 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:18.160600 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6ds4g\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:27:18.162598 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:18.162566 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:18.164250 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.164225 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6"] Jul 15 10:27:18.174307 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.174285 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb"] Jul 15 10:27:18.174879 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.174859 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" containerName="manager" Jul 15 10:27:18.175018 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.174883 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" containerName="manager" Jul 15 10:27:18.175018 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.174996 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" containerName="manager" Jul 15 10:27:18.179552 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.179532 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" Jul 15 10:27:18.193679 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.193655 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb"] Jul 15 10:27:18.216934 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:18.216675 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6ds4g\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:27:18.218629 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:18.218600 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:18.307454 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.307422 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kntwh\" (UniqueName: \"kubernetes.io/projected/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-kube-api-access-kntwh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v7rmv\" (UID: \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:18.307606 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.307511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v7rmv\" (UID: \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:18.307606 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.307532 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n2x5\" (UniqueName: \"kubernetes.io/projected/d47436eb-52dc-409c-bec7-aff61b19f537-kube-api-access-5n2x5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8sn6\" (UID: \"d47436eb-52dc-409c-bec7-aff61b19f537\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:18.307726 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.307665 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d47436eb-52dc-409c-bec7-aff61b19f537-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8sn6\" (UID: \"d47436eb-52dc-409c-bec7-aff61b19f537\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:18.307726 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.307703 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk4nn\" (UniqueName: \"kubernetes.io/projected/dbee81e3-1d56-4b0e-95e5-810243318d30-kube-api-access-bk4nn\") pod \"limitador-operator-controller-manager-859d7b78bd-hdkqb\" (UID: \"dbee81e3-1d56-4b0e-95e5-810243318d30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" Jul 15 10:27:18.409180 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.409100 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d47436eb-52dc-409c-bec7-aff61b19f537-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8sn6\" (UID: \"d47436eb-52dc-409c-bec7-aff61b19f537\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:18.409180 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.409141 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk4nn\" (UniqueName: \"kubernetes.io/projected/dbee81e3-1d56-4b0e-95e5-810243318d30-kube-api-access-bk4nn\") pod \"limitador-operator-controller-manager-859d7b78bd-hdkqb\" (UID: \"dbee81e3-1d56-4b0e-95e5-810243318d30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" Jul 15 10:27:18.409643 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.409183 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntwh\" (UniqueName: \"kubernetes.io/projected/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-kube-api-access-kntwh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v7rmv\" (UID: \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:18.409643 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.409245 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v7rmv\" (UID: \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:18.409643 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.409272 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n2x5\" (UniqueName: \"kubernetes.io/projected/d47436eb-52dc-409c-bec7-aff61b19f537-kube-api-access-5n2x5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8sn6\" (UID: \"d47436eb-52dc-409c-bec7-aff61b19f537\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:18.409643 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.409503 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d47436eb-52dc-409c-bec7-aff61b19f537-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8sn6\" (UID: \"d47436eb-52dc-409c-bec7-aff61b19f537\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:18.409859 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.409672 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v7rmv\" (UID: \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:18.421701 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.421639 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk4nn\" (UniqueName: \"kubernetes.io/projected/dbee81e3-1d56-4b0e-95e5-810243318d30-kube-api-access-bk4nn\") pod \"limitador-operator-controller-manager-859d7b78bd-hdkqb\" (UID: \"dbee81e3-1d56-4b0e-95e5-810243318d30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" Jul 15 10:27:18.421836 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.421724 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n2x5\" (UniqueName: \"kubernetes.io/projected/d47436eb-52dc-409c-bec7-aff61b19f537-kube-api-access-5n2x5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8sn6\" (UID: \"d47436eb-52dc-409c-bec7-aff61b19f537\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:18.421836 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.421782 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kntwh\" (UniqueName: \"kubernetes.io/projected/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-kube-api-access-kntwh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v7rmv\" (UID: \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:18.436477 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.436454 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:18.542423 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.542382 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:18.550799 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:18.550777 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" Jul 15 10:27:29.162594 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.161402 2585 generic.go:356] "Generic (PLEG): container finished" podID="2084243c-3b01-4d8a-a47e-000083f65f0b" containerID="db275146878fe6f5b49a9c88abf9d95db1a3a1d147005d76b18a8ee3b0d6d292" exitCode=0 Jul 15 10:27:29.165297 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.165012 2585 generic.go:356] "Generic (PLEG): container finished" podID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" containerID="5bee5062e241022b198766ca6bb5d12f32f093292f56dc06810d08a43c8adf57" exitCode=0 Jul 15 10:27:29.304963 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.304935 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6"] Jul 15 10:27:29.368813 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:27:29.368743 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47436eb_52dc_409c_bec7_aff61b19f537.slice/crio-2564fcb2f693d95d288201f9f23984820e630d1d670da55b731d132f715c1ec5 WatchSource:0}: Error finding container 2564fcb2f693d95d288201f9f23984820e630d1d670da55b731d132f715c1ec5: Status 404 returned error can't find the container with id 2564fcb2f693d95d288201f9f23984820e630d1d670da55b731d132f715c1ec5 Jul 15 10:27:29.398003 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.397981 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:29.400214 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:29.400191 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:29.426279 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.426260 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:27:29.428394 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:29.428369 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:29.430199 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:29.430171 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6ds4g\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:27:29.515277 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.515253 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnqvn\" (UniqueName: \"kubernetes.io/projected/2084243c-3b01-4d8a-a47e-000083f65f0b-kube-api-access-xnqvn\") pod \"2084243c-3b01-4d8a-a47e-000083f65f0b\" (UID: \"2084243c-3b01-4d8a-a47e-000083f65f0b\") " Jul 15 10:27:29.515363 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.515282 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-extensions-socket-volume\") pod \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\" (UID: \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\") " Jul 15 10:27:29.515446 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.515423 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-645wb\" (UniqueName: \"kubernetes.io/projected/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-kube-api-access-645wb\") pod \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\" (UID: \"dfa37c34-b5cb-445a-a111-5d0e9838f8b5\") " Jul 15 10:27:29.515795 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.515767 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dfa37c34-b5cb-445a-a111-5d0e9838f8b5" (UID: "dfa37c34-b5cb-445a-a111-5d0e9838f8b5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:27:29.517209 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.517185 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2084243c-3b01-4d8a-a47e-000083f65f0b-kube-api-access-xnqvn" (OuterVolumeSpecName: "kube-api-access-xnqvn") pod "2084243c-3b01-4d8a-a47e-000083f65f0b" (UID: "2084243c-3b01-4d8a-a47e-000083f65f0b"). InnerVolumeSpecName "kube-api-access-xnqvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:27:29.517331 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.517306 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-kube-api-access-645wb" (OuterVolumeSpecName: "kube-api-access-645wb") pod "dfa37c34-b5cb-445a-a111-5d0e9838f8b5" (UID: "dfa37c34-b5cb-445a-a111-5d0e9838f8b5"). InnerVolumeSpecName "kube-api-access-645wb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:27:29.547322 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.547304 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb"] Jul 15 10:27:29.549358 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:27:29.549332 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbee81e3_1d56_4b0e_95e5_810243318d30.slice/crio-8b3f7c691ea8f05aae3196ef7daa6ad17415c36d8c5097b5b1aeca465b068c20 WatchSource:0}: Error finding container 8b3f7c691ea8f05aae3196ef7daa6ad17415c36d8c5097b5b1aeca465b068c20: Status 404 returned error can't find the container with id 8b3f7c691ea8f05aae3196ef7daa6ad17415c36d8c5097b5b1aeca465b068c20 Jul 15 10:27:29.550980 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.550960 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv"] Jul 15 10:27:29.552750 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:27:29.552730 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffadec0_a0ab_47ab_9bb1_b9e2defe2c20.slice/crio-a0904ccc4b879bceda0699e9948cd772bf50d330ba6321d16d9029df6806d61d WatchSource:0}: Error finding container a0904ccc4b879bceda0699e9948cd772bf50d330ba6321d16d9029df6806d61d: Status 404 returned error can't find the container with id a0904ccc4b879bceda0699e9948cd772bf50d330ba6321d16d9029df6806d61d Jul 15 10:27:29.616466 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.616427 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-645wb\" (UniqueName: \"kubernetes.io/projected/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-kube-api-access-645wb\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:29.616466 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.616449 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnqvn\" (UniqueName: \"kubernetes.io/projected/2084243c-3b01-4d8a-a47e-000083f65f0b-kube-api-access-xnqvn\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:29.616466 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:29.616462 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa37c34-b5cb-445a-a111-5d0e9838f8b5-extensions-socket-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:30.170977 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.170938 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" event={"ID":"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20","Type":"ContainerStarted","Data":"88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644"} Jul 15 10:27:30.171344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.171108 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:30.171344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.171162 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" event={"ID":"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20","Type":"ContainerStarted","Data":"a0904ccc4b879bceda0699e9948cd772bf50d330ba6321d16d9029df6806d61d"} Jul 15 10:27:30.172226 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.172212 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:27:30.172226 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.172221 2585 scope.go:117] "RemoveContainer" containerID="db275146878fe6f5b49a9c88abf9d95db1a3a1d147005d76b18a8ee3b0d6d292" Jul 15 10:27:30.172942 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:30.172917 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:30.173817 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.173795 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:30.174603 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:30.174579 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6ds4g\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:27:30.175510 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.175488 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" event={"ID":"d47436eb-52dc-409c-bec7-aff61b19f537","Type":"ContainerStarted","Data":"8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69"} Jul 15 10:27:30.175615 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.175536 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:30.175615 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.175558 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" event={"ID":"d47436eb-52dc-409c-bec7-aff61b19f537","Type":"ContainerStarted","Data":"2564fcb2f693d95d288201f9f23984820e630d1d670da55b731d132f715c1ec5"} Jul 15 10:27:30.177221 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.177198 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" event={"ID":"dbee81e3-1d56-4b0e-95e5-810243318d30","Type":"ContainerStarted","Data":"a4d19f498bd6a38d86aacc98db35f61db5c29742c92db489457696d4ad11b3ac"} Jul 15 10:27:30.177317 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.177241 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" event={"ID":"dbee81e3-1d56-4b0e-95e5-810243318d30","Type":"ContainerStarted","Data":"8b3f7c691ea8f05aae3196ef7daa6ad17415c36d8c5097b5b1aeca465b068c20"} Jul 15 10:27:30.177381 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.177351 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" Jul 15 10:27:30.178750 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.178732 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" event={"ID":"30ad177f-7132-4eae-bdc3-69dd28acf490","Type":"ContainerStarted","Data":"8a645ceeb8390be73a9e846ea20fd3f99813cfd866c2a919db29c5a26a07b09f"} Jul 15 10:27:30.182500 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.182468 2585 scope.go:117] "RemoveContainer" containerID="5bee5062e241022b198766ca6bb5d12f32f093292f56dc06810d08a43c8adf57" Jul 15 10:27:30.202600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.202556 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" podStartSLOduration=12.202540597 podStartE2EDuration="12.202540597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:27:30.201614357 +0000 UTC m=+453.717606836" watchObservedRunningTime="2026-07-15 10:27:30.202540597 +0000 UTC m=+453.718533078" Jul 15 10:27:30.224590 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.224549 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" podStartSLOduration=12.224535732 podStartE2EDuration="12.224535732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:27:30.224131004 +0000 UTC m=+453.740123490" watchObservedRunningTime="2026-07-15 10:27:30.224535732 +0000 UTC m=+453.740528216" Jul 15 10:27:30.245145 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.245104 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-44lpc" podStartSLOduration=2.24067642 podStartE2EDuration="24.245093512s" totalImagesPullingTime="22.004417092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:27:30.244387462 +0000 UTC m=+453.760379948" watchObservedRunningTime="2026-07-15 10:27:30.245093512 +0000 UTC m=+453.761085997" Jul 15 10:27:30.266400 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.266365 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" podStartSLOduration=12.266352509 podStartE2EDuration="12.266352509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:27:30.264780522 +0000 UTC m=+453.780773007" watchObservedRunningTime="2026-07-15 10:27:30.266352509 +0000 UTC m=+453.782344994" Jul 15 10:27:30.267024 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:30.267001 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pbsf2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbsf2" Jul 15 10:27:30.268853 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:30.268832 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6ds4g\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ds4g" Jul 15 10:27:30.935226 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.935191 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2084243c-3b01-4d8a-a47e-000083f65f0b" path="/var/lib/kubelet/pods/2084243c-3b01-4d8a-a47e-000083f65f0b/volumes" Jul 15 10:27:30.935567 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:30.935554 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfa37c34-b5cb-445a-a111-5d0e9838f8b5" path="/var/lib/kubelet/pods/dfa37c34-b5cb-445a-a111-5d0e9838f8b5/volumes" Jul 15 10:27:31.674344 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:31.674311 2585 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-hb8c2_kuadrant-system_af7c6127-d72b-438b-bd87-efe2e08f3811_0(9ffb8a81fe92c83e1729922b845f1611ca379b424fc023812850ad7755ffdc45): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-hb8c2 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 15 10:27:31.674600 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:31.674365 2585 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-hb8c2_kuadrant-system_af7c6127-d72b-438b-bd87-efe2e08f3811_0(9ffb8a81fe92c83e1729922b845f1611ca379b424fc023812850ad7755ffdc45): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-hb8c2 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:32.190686 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.190651 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:32.193026 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:32.192999 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="af7c6127-d72b-438b-bd87-efe2e08f3811" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:32.238429 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.238406 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:32.242463 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:32.242439 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="af7c6127-d72b-438b-bd87-efe2e08f3811" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:32.340120 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.340099 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-998vm\" (UniqueName: \"kubernetes.io/projected/af7c6127-d72b-438b-bd87-efe2e08f3811-kube-api-access-998vm\") pod \"af7c6127-d72b-438b-bd87-efe2e08f3811\" (UID: \"af7c6127-d72b-438b-bd87-efe2e08f3811\") " Jul 15 10:27:32.340249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.340128 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af7c6127-d72b-438b-bd87-efe2e08f3811-extensions-socket-volume\") pod \"af7c6127-d72b-438b-bd87-efe2e08f3811\" (UID: \"af7c6127-d72b-438b-bd87-efe2e08f3811\") " Jul 15 10:27:32.340448 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.340427 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7c6127-d72b-438b-bd87-efe2e08f3811-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af7c6127-d72b-438b-bd87-efe2e08f3811" (UID: "af7c6127-d72b-438b-bd87-efe2e08f3811"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:27:32.342199 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.342177 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af7c6127-d72b-438b-bd87-efe2e08f3811-kube-api-access-998vm" (OuterVolumeSpecName: "kube-api-access-998vm") pod "af7c6127-d72b-438b-bd87-efe2e08f3811" (UID: "af7c6127-d72b-438b-bd87-efe2e08f3811"). InnerVolumeSpecName "kube-api-access-998vm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:27:32.440914 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.440856 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-998vm\" (UniqueName: \"kubernetes.io/projected/af7c6127-d72b-438b-bd87-efe2e08f3811-kube-api-access-998vm\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:32.440914 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.440876 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af7c6127-d72b-438b-bd87-efe2e08f3811-extensions-socket-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:32.933514 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:32.933481 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af7c6127-d72b-438b-bd87-efe2e08f3811" path="/var/lib/kubelet/pods/af7c6127-d72b-438b-bd87-efe2e08f3811/volumes" Jul 15 10:27:33.195095 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:33.195015 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:33.197425 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:33.197396 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="af7c6127-d72b-438b-bd87-efe2e08f3811" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:33.199349 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:33.199321 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-hb8c2\" is forbidden: User \"system:node:ip-10-0-130-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-114.ec2.internal' and this object" podUID="af7c6127-d72b-438b-bd87-efe2e08f3811" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hb8c2" Jul 15 10:27:41.187851 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.187815 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdkqb" Jul 15 10:27:41.188360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.187917 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:41.188360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.188348 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:41.247558 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.247529 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6"] Jul 15 10:27:41.247717 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.247699 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" podUID="d47436eb-52dc-409c-bec7-aff61b19f537" containerName="manager" containerID="cri-o://8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69" gracePeriod=10 Jul 15 10:27:41.510870 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.510849 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:41.609239 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.609210 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d47436eb-52dc-409c-bec7-aff61b19f537-extensions-socket-volume\") pod \"d47436eb-52dc-409c-bec7-aff61b19f537\" (UID: \"d47436eb-52dc-409c-bec7-aff61b19f537\") " Jul 15 10:27:41.609374 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.609243 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5n2x5\" (UniqueName: \"kubernetes.io/projected/d47436eb-52dc-409c-bec7-aff61b19f537-kube-api-access-5n2x5\") pod \"d47436eb-52dc-409c-bec7-aff61b19f537\" (UID: \"d47436eb-52dc-409c-bec7-aff61b19f537\") " Jul 15 10:27:41.609552 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.609529 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47436eb-52dc-409c-bec7-aff61b19f537-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d47436eb-52dc-409c-bec7-aff61b19f537" (UID: "d47436eb-52dc-409c-bec7-aff61b19f537"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:27:41.611073 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.611041 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47436eb-52dc-409c-bec7-aff61b19f537-kube-api-access-5n2x5" (OuterVolumeSpecName: "kube-api-access-5n2x5") pod "d47436eb-52dc-409c-bec7-aff61b19f537" (UID: "d47436eb-52dc-409c-bec7-aff61b19f537"). InnerVolumeSpecName "kube-api-access-5n2x5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:27:41.627963 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.627941 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c"] Jul 15 10:27:41.628344 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.628331 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47436eb-52dc-409c-bec7-aff61b19f537" containerName="manager" Jul 15 10:27:41.628396 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.628346 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47436eb-52dc-409c-bec7-aff61b19f537" containerName="manager" Jul 15 10:27:41.628437 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.628416 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47436eb-52dc-409c-bec7-aff61b19f537" containerName="manager" Jul 15 10:27:41.630327 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.630313 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:41.645160 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.645140 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c"] Jul 15 10:27:41.710140 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.710116 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f532d4-ae65-4ccd-a6c7-5d8860e84020-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phm6c\" (UID: \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:41.710231 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.710152 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8l6p\" (UniqueName: \"kubernetes.io/projected/09f532d4-ae65-4ccd-a6c7-5d8860e84020-kube-api-access-l8l6p\") pod \"kuadrant-operator-controller-manager-86878c769f-phm6c\" (UID: \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:41.710308 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.710286 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d47436eb-52dc-409c-bec7-aff61b19f537-extensions-socket-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:41.710352 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.710315 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5n2x5\" (UniqueName: \"kubernetes.io/projected/d47436eb-52dc-409c-bec7-aff61b19f537-kube-api-access-5n2x5\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:41.811270 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.811218 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f532d4-ae65-4ccd-a6c7-5d8860e84020-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phm6c\" (UID: \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:41.811270 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.811260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8l6p\" (UniqueName: \"kubernetes.io/projected/09f532d4-ae65-4ccd-a6c7-5d8860e84020-kube-api-access-l8l6p\") pod \"kuadrant-operator-controller-manager-86878c769f-phm6c\" (UID: \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:41.811555 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.811539 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f532d4-ae65-4ccd-a6c7-5d8860e84020-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phm6c\" (UID: \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:41.821935 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.821911 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8l6p\" (UniqueName: \"kubernetes.io/projected/09f532d4-ae65-4ccd-a6c7-5d8860e84020-kube-api-access-l8l6p\") pod \"kuadrant-operator-controller-manager-86878c769f-phm6c\" (UID: \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:41.940252 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:41.940222 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:42.069438 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.069411 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c"] Jul 15 10:27:42.070950 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:27:42.070923 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f532d4_ae65_4ccd_a6c7_5d8860e84020.slice/crio-b3a13a6b423814e09197371e257cfff3b58b16e88a1f890f6c8fb115d60ec2b7 WatchSource:0}: Error finding container b3a13a6b423814e09197371e257cfff3b58b16e88a1f890f6c8fb115d60ec2b7: Status 404 returned error can't find the container with id b3a13a6b423814e09197371e257cfff3b58b16e88a1f890f6c8fb115d60ec2b7 Jul 15 10:27:42.240556 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.240527 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" event={"ID":"09f532d4-ae65-4ccd-a6c7-5d8860e84020","Type":"ContainerStarted","Data":"6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0"} Jul 15 10:27:42.240967 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.240585 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" event={"ID":"09f532d4-ae65-4ccd-a6c7-5d8860e84020","Type":"ContainerStarted","Data":"b3a13a6b423814e09197371e257cfff3b58b16e88a1f890f6c8fb115d60ec2b7"} Jul 15 10:27:42.240967 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.240618 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:42.241831 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.241811 2585 generic.go:356] "Generic (PLEG): container finished" podID="d47436eb-52dc-409c-bec7-aff61b19f537" containerID="8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69" exitCode=0 Jul 15 10:27:42.241919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.241850 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" Jul 15 10:27:42.241919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.241875 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" event={"ID":"d47436eb-52dc-409c-bec7-aff61b19f537","Type":"ContainerDied","Data":"8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69"} Jul 15 10:27:42.241919 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.241909 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6" event={"ID":"d47436eb-52dc-409c-bec7-aff61b19f537","Type":"ContainerDied","Data":"2564fcb2f693d95d288201f9f23984820e630d1d670da55b731d132f715c1ec5"} Jul 15 10:27:42.242049 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.241927 2585 scope.go:117] "RemoveContainer" containerID="8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69" Jul 15 10:27:42.250978 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.250913 2585 scope.go:117] "RemoveContainer" containerID="8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69" Jul 15 10:27:42.251220 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:42.251198 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69\": container with ID starting with 8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69 not found: ID does not exist" containerID="8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69" Jul 15 10:27:42.251288 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.251231 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69"} err="failed to get container status \"8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69\": rpc error: code = NotFound desc = could not find container \"8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69\": container with ID starting with 8b6f9947ced7a8e1b4b87256ad04abf0fa8ba8377fb0914142a8a7b3a8fdce69 not found: ID does not exist" Jul 15 10:27:42.276810 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.272480 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" podStartSLOduration=1.272464341 podStartE2EDuration="1.272464341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:27:42.27041325 +0000 UTC m=+465.786405735" watchObservedRunningTime="2026-07-15 10:27:42.272464341 +0000 UTC m=+465.788456825" Jul 15 10:27:42.293740 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.293715 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6"] Jul 15 10:27:42.296549 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.296530 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8sn6"] Jul 15 10:27:42.933483 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:42.933450 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d47436eb-52dc-409c-bec7-aff61b19f537" path="/var/lib/kubelet/pods/d47436eb-52dc-409c-bec7-aff61b19f537/volumes" Jul 15 10:27:53.251826 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.251794 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:27:53.316332 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.316303 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv"] Jul 15 10:27:53.316550 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.316510 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" podUID="1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20" containerName="manager" containerID="cri-o://88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644" gracePeriod=10 Jul 15 10:27:53.558629 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.558607 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:53.608133 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.608110 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-extensions-socket-volume\") pod \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\" (UID: \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\") " Jul 15 10:27:53.608245 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.608164 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kntwh\" (UniqueName: \"kubernetes.io/projected/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-kube-api-access-kntwh\") pod \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\" (UID: \"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20\") " Jul 15 10:27:53.608448 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.608429 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20" (UID: "1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:27:53.610027 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.610000 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-kube-api-access-kntwh" (OuterVolumeSpecName: "kube-api-access-kntwh") pod "1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20" (UID: "1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20"). InnerVolumeSpecName "kube-api-access-kntwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:27:53.709175 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.709151 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-extensions-socket-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:53.709175 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:53.709172 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kntwh\" (UniqueName: \"kubernetes.io/projected/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20-kube-api-access-kntwh\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:27:54.306783 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.306752 2585 generic.go:356] "Generic (PLEG): container finished" podID="1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20" containerID="88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644" exitCode=0 Jul 15 10:27:54.307238 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.306797 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" Jul 15 10:27:54.307238 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.306824 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" event={"ID":"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20","Type":"ContainerDied","Data":"88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644"} Jul 15 10:27:54.307238 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.306884 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv" event={"ID":"1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20","Type":"ContainerDied","Data":"a0904ccc4b879bceda0699e9948cd772bf50d330ba6321d16d9029df6806d61d"} Jul 15 10:27:54.307238 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.306918 2585 scope.go:117] "RemoveContainer" containerID="88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644" Jul 15 10:27:54.318266 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.318096 2585 scope.go:117] "RemoveContainer" containerID="88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644" Jul 15 10:27:54.318412 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:27:54.318387 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644\": container with ID starting with 88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644 not found: ID does not exist" containerID="88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644" Jul 15 10:27:54.318465 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.318426 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644"} err="failed to get container status \"88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644\": rpc error: code = NotFound desc = could not find container \"88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644\": container with ID starting with 88477f83eb5cf2e5ba72ff6f306b3a235f10b428de36195f4c474432c3058644 not found: ID does not exist" Jul 15 10:27:54.334505 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.334483 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv"] Jul 15 10:27:54.342039 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.342010 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v7rmv"] Jul 15 10:27:54.934475 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:54.934444 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20" path="/var/lib/kubelet/pods/1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20/volumes" Jul 15 10:27:57.750249 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:57.750213 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq"] Jul 15 10:27:57.750607 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:57.750595 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20" containerName="manager" Jul 15 10:27:57.750650 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:57.750608 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20" containerName="manager" Jul 15 10:27:57.750685 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:57.750667 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ffadec0-a0ab-47ab-9bb1-b9e2defe2c20" containerName="manager" Jul 15 10:27:57.944601 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:57.944566 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq"] Jul 15 10:27:57.944928 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:57.944907 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:57.948152 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:57.948125 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tr5vz\"" Jul 15 10:27:57.948152 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:57.948146 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 10:27:58.039922 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.039865 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.039922 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.039906 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.040119 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.039932 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.040119 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.040004 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.040119 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.040030 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1c4e79-589f-431c-96fb-63e32a17517b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.040119 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.040094 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.040119 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.040111 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th292\" (UniqueName: \"kubernetes.io/projected/8b1c4e79-589f-431c-96fb-63e32a17517b-kube-api-access-th292\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.040290 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.040147 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.040290 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.040194 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.140901 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.140876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.140901 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.140902 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1c4e79-589f-431c-96fb-63e32a17517b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141056 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.140931 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141056 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.140947 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th292\" (UniqueName: \"kubernetes.io/projected/8b1c4e79-589f-431c-96fb-63e32a17517b-kube-api-access-th292\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141056 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.140964 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141056 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141011 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141275 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141058 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141275 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141275 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141223 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141292 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141397 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141362 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141609 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141591 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141680 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141633 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.141680 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.141655 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1c4e79-589f-431c-96fb-63e32a17517b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.142985 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.142963 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.143327 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.143309 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.152297 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.152272 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1c4e79-589f-431c-96fb-63e32a17517b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.152546 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.152528 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th292\" (UniqueName: \"kubernetes.io/projected/8b1c4e79-589f-431c-96fb-63e32a17517b-kube-api-access-th292\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7bdxq\" (UID: \"8b1c4e79-589f-431c-96fb-63e32a17517b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.254166 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.254143 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:27:58.403026 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:58.403002 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq"] Jul 15 10:27:58.404439 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:27:58.404413 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1c4e79_589f_431c_96fb_63e32a17517b.slice/crio-4c5e5738431f6473cfd1f0948ff5882e67fe7e51c23e832183d275e376ebc4bb WatchSource:0}: Error finding container 4c5e5738431f6473cfd1f0948ff5882e67fe7e51c23e832183d275e376ebc4bb: Status 404 returned error can't find the container with id 4c5e5738431f6473cfd1f0948ff5882e67fe7e51c23e832183d275e376ebc4bb Jul 15 10:27:59.333440 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:27:59.333400 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" event={"ID":"8b1c4e79-589f-431c-96fb-63e32a17517b","Type":"ContainerStarted","Data":"4c5e5738431f6473cfd1f0948ff5882e67fe7e51c23e832183d275e376ebc4bb"} Jul 15 10:28:02.564950 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:02.564916 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:28:02.565239 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:02.564983 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:28:02.565239 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:02.565015 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:28:03.355371 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:03.355335 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" event={"ID":"8b1c4e79-589f-431c-96fb-63e32a17517b","Type":"ContainerStarted","Data":"6e4c76e03d49c2d8858b764b4bce999533db2133bc9d7958a5955aa6cb78ffb0"} Jul 15 10:28:03.386644 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:03.386598 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" podStartSLOduration=2.228407228 podStartE2EDuration="6.386582146s" totalImagesPullingTime="4.158174918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:28:03.385567115 +0000 UTC m=+486.901559600" watchObservedRunningTime="2026-07-15 10:28:03.386582146 +0000 UTC m=+486.902574631" Jul 15 10:28:04.254563 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:04.254534 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:28:04.259371 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:04.259342 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:28:04.359701 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:04.359675 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:28:04.360354 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:04.360333 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:28:12.965315 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:12.965279 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mmcd8"] Jul 15 10:28:12.970105 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:12.970080 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:12.972569 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:12.972548 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 10:28:12.979594 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:12.979571 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mmcd8"] Jul 15 10:28:13.056457 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.056421 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml2w9\" (UniqueName: \"kubernetes.io/projected/7620f026-ad9d-4c24-b47b-307e2e8a0e22-kube-api-access-ml2w9\") pod \"limitador-limitador-64c94fd64f-mmcd8\" (UID: \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:13.056624 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.056485 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7620f026-ad9d-4c24-b47b-307e2e8a0e22-config-file\") pod \"limitador-limitador-64c94fd64f-mmcd8\" (UID: \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:13.062645 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.062619 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mmcd8"] Jul 15 10:28:13.158029 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.158005 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml2w9\" (UniqueName: \"kubernetes.io/projected/7620f026-ad9d-4c24-b47b-307e2e8a0e22-kube-api-access-ml2w9\") pod \"limitador-limitador-64c94fd64f-mmcd8\" (UID: \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:13.158219 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.158041 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7620f026-ad9d-4c24-b47b-307e2e8a0e22-config-file\") pod \"limitador-limitador-64c94fd64f-mmcd8\" (UID: \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:13.158641 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.158623 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7620f026-ad9d-4c24-b47b-307e2e8a0e22-config-file\") pod \"limitador-limitador-64c94fd64f-mmcd8\" (UID: \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:13.169922 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.169894 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml2w9\" (UniqueName: \"kubernetes.io/projected/7620f026-ad9d-4c24-b47b-307e2e8a0e22-kube-api-access-ml2w9\") pod \"limitador-limitador-64c94fd64f-mmcd8\" (UID: \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:13.279695 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.279670 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:13.415915 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:13.415891 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mmcd8"] Jul 15 10:28:13.418487 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:28:13.418456 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7620f026_ad9d_4c24_b47b_307e2e8a0e22.slice/crio-d0488f2ac770afcfaa36450dcc0821421e34a1497a4fb6d3011748344da41fc6 WatchSource:0}: Error finding container d0488f2ac770afcfaa36450dcc0821421e34a1497a4fb6d3011748344da41fc6: Status 404 returned error can't find the container with id d0488f2ac770afcfaa36450dcc0821421e34a1497a4fb6d3011748344da41fc6 Jul 15 10:28:14.416525 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:14.416485 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" event={"ID":"7620f026-ad9d-4c24-b47b-307e2e8a0e22","Type":"ContainerStarted","Data":"d0488f2ac770afcfaa36450dcc0821421e34a1497a4fb6d3011748344da41fc6"} Jul 15 10:28:17.433152 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:17.433120 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" event={"ID":"7620f026-ad9d-4c24-b47b-307e2e8a0e22","Type":"ContainerStarted","Data":"92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4"} Jul 15 10:28:17.433517 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:17.433189 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:17.454075 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:17.454018 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" podStartSLOduration=2.458692555 podStartE2EDuration="5.454003495s" totalImagesPullingTime="2.99531094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:28:17.452612099 +0000 UTC m=+500.968604586" watchObservedRunningTime="2026-07-15 10:28:17.454003495 +0000 UTC m=+500.969996041" Jul 15 10:28:28.110557 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.110519 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mmcd8"] Jul 15 10:28:28.110987 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.110864 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" podUID="7620f026-ad9d-4c24-b47b-307e2e8a0e22" containerName="limitador" containerID="cri-o://92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4" gracePeriod=30 Jul 15 10:28:28.116726 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.116530 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:28.660360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.660337 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:28.781525 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.781500 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml2w9\" (UniqueName: \"kubernetes.io/projected/7620f026-ad9d-4c24-b47b-307e2e8a0e22-kube-api-access-ml2w9\") pod \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\" (UID: \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\") " Jul 15 10:28:28.781655 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.781563 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7620f026-ad9d-4c24-b47b-307e2e8a0e22-config-file\") pod \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\" (UID: \"7620f026-ad9d-4c24-b47b-307e2e8a0e22\") " Jul 15 10:28:28.781932 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.781904 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7620f026-ad9d-4c24-b47b-307e2e8a0e22-config-file" (OuterVolumeSpecName: "config-file") pod "7620f026-ad9d-4c24-b47b-307e2e8a0e22" (UID: "7620f026-ad9d-4c24-b47b-307e2e8a0e22"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:28:28.783591 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.783482 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7620f026-ad9d-4c24-b47b-307e2e8a0e22-kube-api-access-ml2w9" (OuterVolumeSpecName: "kube-api-access-ml2w9") pod "7620f026-ad9d-4c24-b47b-307e2e8a0e22" (UID: "7620f026-ad9d-4c24-b47b-307e2e8a0e22"). InnerVolumeSpecName "kube-api-access-ml2w9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:28:28.882595 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.882571 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml2w9\" (UniqueName: \"kubernetes.io/projected/7620f026-ad9d-4c24-b47b-307e2e8a0e22-kube-api-access-ml2w9\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:28:28.882595 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:28.882596 2585 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7620f026-ad9d-4c24-b47b-307e2e8a0e22-config-file\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:28:29.493768 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.493737 2585 generic.go:356] "Generic (PLEG): container finished" podID="7620f026-ad9d-4c24-b47b-307e2e8a0e22" containerID="92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4" exitCode=0 Jul 15 10:28:29.494198 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.493780 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" Jul 15 10:28:29.494198 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.493792 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" event={"ID":"7620f026-ad9d-4c24-b47b-307e2e8a0e22","Type":"ContainerDied","Data":"92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4"} Jul 15 10:28:29.494198 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.493836 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mmcd8" event={"ID":"7620f026-ad9d-4c24-b47b-307e2e8a0e22","Type":"ContainerDied","Data":"d0488f2ac770afcfaa36450dcc0821421e34a1497a4fb6d3011748344da41fc6"} Jul 15 10:28:29.494198 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.493865 2585 scope.go:117] "RemoveContainer" containerID="92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4" Jul 15 10:28:29.502621 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.502590 2585 scope.go:117] "RemoveContainer" containerID="92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4" Jul 15 10:28:29.502882 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:28:29.502859 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4\": container with ID starting with 92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4 not found: ID does not exist" containerID="92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4" Jul 15 10:28:29.502955 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.502895 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4"} err="failed to get container status \"92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4\": rpc error: code = NotFound desc = could not find container \"92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4\": container with ID starting with 92f128a72fdb3a28581c7badc9e13e07f2df49dcea83a96fc9c48e82f7aca8c4 not found: ID does not exist" Jul 15 10:28:29.511967 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.511931 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mmcd8"] Jul 15 10:28:29.519095 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:29.519050 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mmcd8"] Jul 15 10:28:30.934528 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:30.934486 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7620f026-ad9d-4c24-b47b-307e2e8a0e22" path="/var/lib/kubelet/pods/7620f026-ad9d-4c24-b47b-307e2e8a0e22/volumes" Jul 15 10:28:31.490583 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.490551 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s"] Jul 15 10:28:31.491142 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.491122 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7620f026-ad9d-4c24-b47b-307e2e8a0e22" containerName="limitador" Jul 15 10:28:31.491194 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.491146 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7620f026-ad9d-4c24-b47b-307e2e8a0e22" containerName="limitador" Jul 15 10:28:31.491291 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.491278 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7620f026-ad9d-4c24-b47b-307e2e8a0e22" containerName="limitador" Jul 15 10:28:31.494368 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.494346 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:31.496668 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.496644 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cwf2r\"" Jul 15 10:28:31.496668 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.496657 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 10:28:31.497267 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.497247 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 10:28:31.497370 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.497255 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 15 10:28:31.503204 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.503051 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s"] Jul 15 10:28:31.603373 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.603348 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/98356cee-41bc-488e-8459-4968c2a00d95-data\") pod \"postgres-69fb4594cf-vgf9s\" (UID: \"98356cee-41bc-488e-8459-4968c2a00d95\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:31.603492 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.603385 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6nx\" (UniqueName: \"kubernetes.io/projected/98356cee-41bc-488e-8459-4968c2a00d95-kube-api-access-ql6nx\") pod \"postgres-69fb4594cf-vgf9s\" (UID: \"98356cee-41bc-488e-8459-4968c2a00d95\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:31.704518 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.704490 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6nx\" (UniqueName: \"kubernetes.io/projected/98356cee-41bc-488e-8459-4968c2a00d95-kube-api-access-ql6nx\") pod \"postgres-69fb4594cf-vgf9s\" (UID: \"98356cee-41bc-488e-8459-4968c2a00d95\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:31.704636 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.704614 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/98356cee-41bc-488e-8459-4968c2a00d95-data\") pod \"postgres-69fb4594cf-vgf9s\" (UID: \"98356cee-41bc-488e-8459-4968c2a00d95\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:31.704948 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.704928 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/98356cee-41bc-488e-8459-4968c2a00d95-data\") pod \"postgres-69fb4594cf-vgf9s\" (UID: \"98356cee-41bc-488e-8459-4968c2a00d95\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:31.714523 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.714497 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6nx\" (UniqueName: \"kubernetes.io/projected/98356cee-41bc-488e-8459-4968c2a00d95-kube-api-access-ql6nx\") pod \"postgres-69fb4594cf-vgf9s\" (UID: \"98356cee-41bc-488e-8459-4968c2a00d95\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:31.806510 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.806463 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:31.931921 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:31.931897 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s"] Jul 15 10:28:31.934354 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:28:31.934326 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98356cee_41bc_488e_8459_4968c2a00d95.slice/crio-f534c78ffc4f63ed448600f4277c174af217f755276047ac633a8242b5e24da3 WatchSource:0}: Error finding container f534c78ffc4f63ed448600f4277c174af217f755276047ac633a8242b5e24da3: Status 404 returned error can't find the container with id f534c78ffc4f63ed448600f4277c174af217f755276047ac633a8242b5e24da3 Jul 15 10:28:32.511097 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:32.511049 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" event={"ID":"98356cee-41bc-488e-8459-4968c2a00d95","Type":"ContainerStarted","Data":"f534c78ffc4f63ed448600f4277c174af217f755276047ac633a8242b5e24da3"} Jul 15 10:28:37.541643 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:37.541593 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" event={"ID":"98356cee-41bc-488e-8459-4968c2a00d95","Type":"ContainerStarted","Data":"8e9c926f216e1adc7e914535909daf8f2cbc21817bc0d1e43228fbea670ca368"} Jul 15 10:28:37.542031 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:37.541678 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:37.564014 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:37.563962 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" podStartSLOduration=1.736896583 podStartE2EDuration="6.563949916s" totalImagesPullingTime="4.827053333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:28:37.561980321 +0000 UTC m=+521.077972806" watchObservedRunningTime="2026-07-15 10:28:37.563949916 +0000 UTC m=+521.079942401" Jul 15 10:28:43.574436 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:43.574410 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vgf9s" Jul 15 10:28:50.449854 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.449783 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-546b84cc7d-8cgh9"] Jul 15 10:28:50.457484 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.457467 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:50.460086 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.460023 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 10:28:50.460206 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.460088 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q6q65\"" Jul 15 10:28:50.460370 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.460353 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 10:28:50.468463 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.468441 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-546b84cc7d-8cgh9"] Jul 15 10:28:50.565127 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.565093 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdl82\" (UniqueName: \"kubernetes.io/projected/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-kube-api-access-rdl82\") pod \"maas-controller-546b84cc7d-8cgh9\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:50.565261 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.565142 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert\") pod \"maas-controller-546b84cc7d-8cgh9\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:50.637970 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.637937 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57b5f89b76-l7zzc"] Jul 15 10:28:50.641741 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.641720 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:50.653697 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.653678 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5f89b76-l7zzc"] Jul 15 10:28:50.666473 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.666452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdl82\" (UniqueName: \"kubernetes.io/projected/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-kube-api-access-rdl82\") pod \"maas-controller-546b84cc7d-8cgh9\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:50.666580 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.666497 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert\") pod \"maas-controller-546b84cc7d-8cgh9\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:50.666643 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:28:50.666624 2585 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 15 10:28:50.666743 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:28:50.666716 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert podName:d2d98d6e-abed-4b4e-b7fa-45597b7767d6 nodeName:}" failed. No retries permitted until 2026-07-15 10:28:51.166668395 +0000 UTC m=+534.682660860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert") pod "maas-controller-546b84cc7d-8cgh9" (UID: "d2d98d6e-abed-4b4e-b7fa-45597b7767d6") : secret "maas-controller-webhook-cert" not found Jul 15 10:28:50.676027 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.676005 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdl82\" (UniqueName: \"kubernetes.io/projected/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-kube-api-access-rdl82\") pod \"maas-controller-546b84cc7d-8cgh9\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:50.767682 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.767655 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gm9r\" (UniqueName: \"kubernetes.io/projected/7ec89c51-2afd-4cdb-8ab2-eceb86083639-kube-api-access-2gm9r\") pod \"maas-controller-57b5f89b76-l7zzc\" (UID: \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\") " pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:50.767767 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.767734 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7ec89c51-2afd-4cdb-8ab2-eceb86083639-webhook-cert\") pod \"maas-controller-57b5f89b76-l7zzc\" (UID: \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\") " pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:50.868196 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.868173 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gm9r\" (UniqueName: \"kubernetes.io/projected/7ec89c51-2afd-4cdb-8ab2-eceb86083639-kube-api-access-2gm9r\") pod \"maas-controller-57b5f89b76-l7zzc\" (UID: \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\") " pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:50.868285 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.868200 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7ec89c51-2afd-4cdb-8ab2-eceb86083639-webhook-cert\") pod \"maas-controller-57b5f89b76-l7zzc\" (UID: \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\") " pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:50.870405 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.870386 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7ec89c51-2afd-4cdb-8ab2-eceb86083639-webhook-cert\") pod \"maas-controller-57b5f89b76-l7zzc\" (UID: \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\") " pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:50.877793 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.877776 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gm9r\" (UniqueName: \"kubernetes.io/projected/7ec89c51-2afd-4cdb-8ab2-eceb86083639-kube-api-access-2gm9r\") pod \"maas-controller-57b5f89b76-l7zzc\" (UID: \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\") " pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:50.951614 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:50.951594 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:51.079758 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:51.079733 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5f89b76-l7zzc"] Jul 15 10:28:51.082747 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:28:51.082726 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec89c51_2afd_4cdb_8ab2_eceb86083639.slice/crio-3d07df5f077a1fde5576108a1b18a4b38ee916d0a22d61a98b2e2c8d09572e86 WatchSource:0}: Error finding container 3d07df5f077a1fde5576108a1b18a4b38ee916d0a22d61a98b2e2c8d09572e86: Status 404 returned error can't find the container with id 3d07df5f077a1fde5576108a1b18a4b38ee916d0a22d61a98b2e2c8d09572e86 Jul 15 10:28:51.171917 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:51.171892 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert\") pod \"maas-controller-546b84cc7d-8cgh9\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:51.173988 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:51.173966 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert\") pod \"maas-controller-546b84cc7d-8cgh9\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:51.368250 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:51.368185 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:51.496839 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:51.496810 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-546b84cc7d-8cgh9"] Jul 15 10:28:51.498714 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:28:51.498689 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d98d6e_abed_4b4e_b7fa_45597b7767d6.slice/crio-63e8a6d35b8e040082d41c74b884e31fe39a77dbba39f20a5d76452f14e8a7a4 WatchSource:0}: Error finding container 63e8a6d35b8e040082d41c74b884e31fe39a77dbba39f20a5d76452f14e8a7a4: Status 404 returned error can't find the container with id 63e8a6d35b8e040082d41c74b884e31fe39a77dbba39f20a5d76452f14e8a7a4 Jul 15 10:28:51.612535 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:51.612500 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" event={"ID":"d2d98d6e-abed-4b4e-b7fa-45597b7767d6","Type":"ContainerStarted","Data":"63e8a6d35b8e040082d41c74b884e31fe39a77dbba39f20a5d76452f14e8a7a4"} Jul 15 10:28:51.613837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:51.613814 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" event={"ID":"7ec89c51-2afd-4cdb-8ab2-eceb86083639","Type":"ContainerStarted","Data":"3d07df5f077a1fde5576108a1b18a4b38ee916d0a22d61a98b2e2c8d09572e86"} Jul 15 10:28:54.634142 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:54.634092 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" event={"ID":"d2d98d6e-abed-4b4e-b7fa-45597b7767d6","Type":"ContainerStarted","Data":"e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606"} Jul 15 10:28:54.634582 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:54.634270 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:28:54.635746 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:54.635717 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" event={"ID":"7ec89c51-2afd-4cdb-8ab2-eceb86083639","Type":"ContainerStarted","Data":"e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4"} Jul 15 10:28:54.635869 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:54.635775 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:28:54.659198 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:54.659148 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" podStartSLOduration=2.079525926 podStartE2EDuration="4.659136229s" totalImagesPullingTime="2.579610303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:28:54.657057016 +0000 UTC m=+538.173049503" watchObservedRunningTime="2026-07-15 10:28:54.659136229 +0000 UTC m=+538.175128714" Jul 15 10:28:54.682922 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:28:54.682878 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" podStartSLOduration=1.676330114 podStartE2EDuration="4.682860966s" totalImagesPullingTime="3.006530852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:28:54.681005807 +0000 UTC m=+538.196998292" watchObservedRunningTime="2026-07-15 10:28:54.682860966 +0000 UTC m=+538.198853451" Jul 15 10:29:05.642754 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:05.642722 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:29:05.643262 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:05.642861 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:29:05.710653 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:05.710623 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-546b84cc7d-8cgh9"] Jul 15 10:29:05.710861 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:05.710839 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" podUID="d2d98d6e-abed-4b4e-b7fa-45597b7767d6" containerName="manager" containerID="cri-o://e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606" gracePeriod=30 Jul 15 10:29:05.958192 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:05.958169 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:29:06.093087 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.093040 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert\") pod \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " Jul 15 10:29:06.093247 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.093137 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdl82\" (UniqueName: \"kubernetes.io/projected/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-kube-api-access-rdl82\") pod \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\" (UID: \"d2d98d6e-abed-4b4e-b7fa-45597b7767d6\") " Jul 15 10:29:06.095121 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.095095 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d2d98d6e-abed-4b4e-b7fa-45597b7767d6" (UID: "d2d98d6e-abed-4b4e-b7fa-45597b7767d6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:29:06.095230 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.095178 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-kube-api-access-rdl82" (OuterVolumeSpecName: "kube-api-access-rdl82") pod "d2d98d6e-abed-4b4e-b7fa-45597b7767d6" (UID: "d2d98d6e-abed-4b4e-b7fa-45597b7767d6"). InnerVolumeSpecName "kube-api-access-rdl82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:29:06.194607 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.194554 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-webhook-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:29:06.194607 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.194577 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdl82\" (UniqueName: \"kubernetes.io/projected/d2d98d6e-abed-4b4e-b7fa-45597b7767d6-kube-api-access-rdl82\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:29:06.697422 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.697392 2585 generic.go:356] "Generic (PLEG): container finished" podID="d2d98d6e-abed-4b4e-b7fa-45597b7767d6" containerID="e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606" exitCode=0 Jul 15 10:29:06.697830 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.697453 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" Jul 15 10:29:06.697830 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.697491 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" event={"ID":"d2d98d6e-abed-4b4e-b7fa-45597b7767d6","Type":"ContainerDied","Data":"e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606"} Jul 15 10:29:06.697830 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.697596 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-546b84cc7d-8cgh9" event={"ID":"d2d98d6e-abed-4b4e-b7fa-45597b7767d6","Type":"ContainerDied","Data":"63e8a6d35b8e040082d41c74b884e31fe39a77dbba39f20a5d76452f14e8a7a4"} Jul 15 10:29:06.697830 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.697631 2585 scope.go:117] "RemoveContainer" containerID="e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606" Jul 15 10:29:06.706964 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.706946 2585 scope.go:117] "RemoveContainer" containerID="e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606" Jul 15 10:29:06.707237 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:29:06.707215 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606\": container with ID starting with e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606 not found: ID does not exist" containerID="e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606" Jul 15 10:29:06.707309 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.707244 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606"} err="failed to get container status \"e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606\": rpc error: code = NotFound desc = could not find container \"e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606\": container with ID starting with e6a7b4adf36ede99e57ad616cdde7dbbc87eb4d7adc5d6bb75aaddc810828606 not found: ID does not exist" Jul 15 10:29:06.720992 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.720970 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-546b84cc7d-8cgh9"] Jul 15 10:29:06.725434 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.725413 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-546b84cc7d-8cgh9"] Jul 15 10:29:06.933187 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:06.933159 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2d98d6e-abed-4b4e-b7fa-45597b7767d6" path="/var/lib/kubelet/pods/d2d98d6e-abed-4b4e-b7fa-45597b7767d6/volumes" Jul 15 10:29:26.379399 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.379369 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf"] Jul 15 10:29:26.379923 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.379902 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d98d6e-abed-4b4e-b7fa-45597b7767d6" containerName="manager" Jul 15 10:29:26.380012 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.379926 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d98d6e-abed-4b4e-b7fa-45597b7767d6" containerName="manager" Jul 15 10:29:26.380117 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.380106 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d98d6e-abed-4b4e-b7fa-45597b7767d6" containerName="manager" Jul 15 10:29:26.384699 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.384679 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.387129 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.387110 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 15 10:29:26.395784 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.395760 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf"] Jul 15 10:29:26.552142 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.552115 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-openshift-service-ca\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.552248 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.552152 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcj5n\" (UniqueName: \"kubernetes.io/projected/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-kube-api-access-dcj5n\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.552298 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.552264 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-maas-api-tls\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.652937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.652871 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-openshift-service-ca\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.652937 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.652913 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcj5n\" (UniqueName: \"kubernetes.io/projected/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-kube-api-access-dcj5n\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.653201 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.652978 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-maas-api-tls\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.653491 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.653473 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-openshift-service-ca\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.655276 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.655256 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-maas-api-tls\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.662645 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.662626 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcj5n\" (UniqueName: \"kubernetes.io/projected/ab4b15a7-5e89-4ba5-8b70-cff7574ce483-kube-api-access-dcj5n\") pod \"maas-api-86c7db645b-nlbvf\" (UID: \"ab4b15a7-5e89-4ba5-8b70-cff7574ce483\") " pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.695622 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.695597 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:26.832863 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:26.832834 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf"] Jul 15 10:29:26.834723 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:29:26.834693 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4b15a7_5e89_4ba5_8b70_cff7574ce483.slice/crio-be6896988882c99ac2ff00d2d0d881893f86b2e1a22afdd63bb39d476c8b6d0c WatchSource:0}: Error finding container be6896988882c99ac2ff00d2d0d881893f86b2e1a22afdd63bb39d476c8b6d0c: Status 404 returned error can't find the container with id be6896988882c99ac2ff00d2d0d881893f86b2e1a22afdd63bb39d476c8b6d0c Jul 15 10:29:27.068954 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.068925 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-mx5h8"] Jul 15 10:29:27.072840 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.072819 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:27.075743 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.075718 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 10:29:27.075837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.075821 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m6njh\"" Jul 15 10:29:27.084030 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.084009 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-mx5h8"] Jul 15 10:29:27.124667 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.124645 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-t9z66"] Jul 15 10:29:27.127583 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.127568 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:27.145321 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.145295 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-t9z66"] Jul 15 10:29:27.258904 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.258873 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/919f6cd6-11e2-4e26-8246-51cea1e225d1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-mx5h8\" (UID: \"919f6cd6-11e2-4e26-8246-51cea1e225d1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:27.259093 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.258935 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjrdn\" (UniqueName: \"kubernetes.io/projected/5cada147-88b7-4860-bf21-49eb141c03fd-kube-api-access-zjrdn\") pod \"payload-pre-processing-74584bb9f8-t9z66\" (UID: \"5cada147-88b7-4860-bf21-49eb141c03fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:27.259093 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.259002 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbgzl\" (UniqueName: \"kubernetes.io/projected/919f6cd6-11e2-4e26-8246-51cea1e225d1-kube-api-access-tbgzl\") pod \"payload-processing-f9b8d9b9b-mx5h8\" (UID: \"919f6cd6-11e2-4e26-8246-51cea1e225d1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:27.259093 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.259048 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cada147-88b7-4860-bf21-49eb141c03fd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-t9z66\" (UID: \"5cada147-88b7-4860-bf21-49eb141c03fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:27.360605 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.360517 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjrdn\" (UniqueName: \"kubernetes.io/projected/5cada147-88b7-4860-bf21-49eb141c03fd-kube-api-access-zjrdn\") pod \"payload-pre-processing-74584bb9f8-t9z66\" (UID: \"5cada147-88b7-4860-bf21-49eb141c03fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:27.360605 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.360578 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbgzl\" (UniqueName: \"kubernetes.io/projected/919f6cd6-11e2-4e26-8246-51cea1e225d1-kube-api-access-tbgzl\") pod \"payload-processing-f9b8d9b9b-mx5h8\" (UID: \"919f6cd6-11e2-4e26-8246-51cea1e225d1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:27.360605 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.360600 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cada147-88b7-4860-bf21-49eb141c03fd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-t9z66\" (UID: \"5cada147-88b7-4860-bf21-49eb141c03fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:27.360868 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.360668 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/919f6cd6-11e2-4e26-8246-51cea1e225d1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-mx5h8\" (UID: \"919f6cd6-11e2-4e26-8246-51cea1e225d1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:27.361487 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.361460 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/919f6cd6-11e2-4e26-8246-51cea1e225d1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-mx5h8\" (UID: \"919f6cd6-11e2-4e26-8246-51cea1e225d1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:27.361589 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.361546 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cada147-88b7-4860-bf21-49eb141c03fd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-t9z66\" (UID: \"5cada147-88b7-4860-bf21-49eb141c03fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:27.371580 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.371557 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjrdn\" (UniqueName: \"kubernetes.io/projected/5cada147-88b7-4860-bf21-49eb141c03fd-kube-api-access-zjrdn\") pod \"payload-pre-processing-74584bb9f8-t9z66\" (UID: \"5cada147-88b7-4860-bf21-49eb141c03fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:27.371728 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.371704 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbgzl\" (UniqueName: \"kubernetes.io/projected/919f6cd6-11e2-4e26-8246-51cea1e225d1-kube-api-access-tbgzl\") pod \"payload-processing-f9b8d9b9b-mx5h8\" (UID: \"919f6cd6-11e2-4e26-8246-51cea1e225d1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:27.384373 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.384349 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:27.437336 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.437291 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:27.552105 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.551981 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-mx5h8"] Jul 15 10:29:27.555346 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:29:27.555316 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919f6cd6_11e2_4e26_8246_51cea1e225d1.slice/crio-28d033acb39fd9f48cdf8212bc244a331344c03fceda6a22a9efff9ae0638885 WatchSource:0}: Error finding container 28d033acb39fd9f48cdf8212bc244a331344c03fceda6a22a9efff9ae0638885: Status 404 returned error can't find the container with id 28d033acb39fd9f48cdf8212bc244a331344c03fceda6a22a9efff9ae0638885 Jul 15 10:29:27.601144 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.601117 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-t9z66"] Jul 15 10:29:27.602326 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:29:27.602295 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cada147_88b7_4860_bf21_49eb141c03fd.slice/crio-0d279dace969dc6281ba9b2153d30b471afa1e375f1c94aeb99c7518303c19f3 WatchSource:0}: Error finding container 0d279dace969dc6281ba9b2153d30b471afa1e375f1c94aeb99c7518303c19f3: Status 404 returned error can't find the container with id 0d279dace969dc6281ba9b2153d30b471afa1e375f1c94aeb99c7518303c19f3 Jul 15 10:29:27.806606 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.806569 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" event={"ID":"5cada147-88b7-4860-bf21-49eb141c03fd","Type":"ContainerStarted","Data":"0d279dace969dc6281ba9b2153d30b471afa1e375f1c94aeb99c7518303c19f3"} Jul 15 10:29:27.807837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.807808 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" event={"ID":"ab4b15a7-5e89-4ba5-8b70-cff7574ce483","Type":"ContainerStarted","Data":"be6896988882c99ac2ff00d2d0d881893f86b2e1a22afdd63bb39d476c8b6d0c"} Jul 15 10:29:27.809127 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:27.809099 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" event={"ID":"919f6cd6-11e2-4e26-8246-51cea1e225d1","Type":"ContainerStarted","Data":"28d033acb39fd9f48cdf8212bc244a331344c03fceda6a22a9efff9ae0638885"} Jul 15 10:29:29.825949 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.825909 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" event={"ID":"ab4b15a7-5e89-4ba5-8b70-cff7574ce483","Type":"ContainerStarted","Data":"f5b61793776decad10906601cacefc1d97a0fff446e7936ebb8306f40009333f"} Jul 15 10:29:29.826719 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.826691 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:29.829416 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.828863 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" event={"ID":"919f6cd6-11e2-4e26-8246-51cea1e225d1","Type":"ContainerStarted","Data":"51c915b661c2f2256052afdb60a418de9f816b4bee963b12ff8a4ec96684b806"} Jul 15 10:29:29.829416 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.829356 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:29.832778 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.832261 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" event={"ID":"5cada147-88b7-4860-bf21-49eb141c03fd","Type":"ContainerStarted","Data":"26237e5af9471319ebc8425556c4c7e6ff9e4522d35307ce209839ee7c4bb167"} Jul 15 10:29:29.832778 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.832720 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:29.848084 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.846628 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" podStartSLOduration=1.29501537 podStartE2EDuration="3.846611646s" totalImagesPullingTime="2.551596276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:29:29.844772084 +0000 UTC m=+573.360764569" watchObservedRunningTime="2026-07-15 10:29:29.846611646 +0000 UTC m=+573.362604130" Jul 15 10:29:29.867525 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.867485 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" podStartSLOduration=1.081287688 podStartE2EDuration="2.867472744s" totalImagesPullingTime="1.786185056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:29:29.866937577 +0000 UTC m=+573.382930062" watchObservedRunningTime="2026-07-15 10:29:29.867472744 +0000 UTC m=+573.383465228" Jul 15 10:29:29.890418 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:29.890382 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" podStartSLOduration=1.060288948 podStartE2EDuration="2.890370753s" totalImagesPullingTime="1.830081805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:29:29.889179107 +0000 UTC m=+573.405171582" watchObservedRunningTime="2026-07-15 10:29:29.890370753 +0000 UTC m=+573.406363237" Jul 15 10:29:34.360733 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:34.360689 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7bdxq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 10:29:34.361081 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:34.360759 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" podUID="8b1c4e79-589f-431c-96fb-63e32a17517b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 10:29:34.862039 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:34.862010 2585 generic.go:356] "Generic (PLEG): container finished" podID="8b1c4e79-589f-431c-96fb-63e32a17517b" containerID="6e4c76e03d49c2d8858b764b4bce999533db2133bc9d7958a5955aa6cb78ffb0" exitCode=0 Jul 15 10:29:34.862203 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:34.862095 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" event={"ID":"8b1c4e79-589f-431c-96fb-63e32a17517b","Type":"ContainerDied","Data":"6e4c76e03d49c2d8858b764b4bce999533db2133bc9d7958a5955aa6cb78ffb0"} Jul 15 10:29:34.862451 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:34.862436 2585 scope.go:117] "RemoveContainer" containerID="6e4c76e03d49c2d8858b764b4bce999533db2133bc9d7958a5955aa6cb78ffb0" Jul 15 10:29:34.863480 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:34.863449 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:29:34.863545 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:34.863528 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:29:34.863581 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:34.863566 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:29:35.868603 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:35.868570 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" event={"ID":"8b1c4e79-589f-431c-96fb-63e32a17517b","Type":"ContainerStarted","Data":"8a699237eff5339906db2ecff373ef29e5c179cf39b4a8d843e4e660aba5a49e"} Jul 15 10:29:36.254938 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:36.254900 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:29:36.255186 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:36.255163 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7bdxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 10:29:36.255327 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:36.255226 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" podUID="8b1c4e79-589f-431c-96fb-63e32a17517b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 10:29:36.852996 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:36.852965 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86c7db645b-nlbvf" Jul 15 10:29:37.254774 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:37.254745 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7bdxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 10:29:37.255174 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:37.254797 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" podUID="8b1c4e79-589f-431c-96fb-63e32a17517b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 10:29:38.254513 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:38.254476 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7bdxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 10:29:38.254692 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:38.254529 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" podUID="8b1c4e79-589f-431c-96fb-63e32a17517b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 10:29:39.255015 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:39.254979 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7bdxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 10:29:39.255413 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:39.255030 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" podUID="8b1c4e79-589f-431c-96fb-63e32a17517b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 10:29:40.258662 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:40.258632 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:29:40.259117 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:40.259006 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:29:40.259764 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:40.259742 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7bdxq" Jul 15 10:29:41.846660 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:41.846633 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-mx5h8" Jul 15 10:29:41.847027 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:41.846731 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-t9z66" Jul 15 10:29:46.175170 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.175135 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld"] Jul 15 10:29:46.178429 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.178410 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.181054 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.181032 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 10:29:46.181894 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.181874 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 15 10:29:46.182029 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.181920 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 10:29:46.182029 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.181976 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lmmjs\"" Jul 15 10:29:46.191147 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.191124 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld"] Jul 15 10:29:46.219595 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.219564 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.219716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.219605 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12920f25-3947-466c-8046-3bc0e4c9f458-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.219716 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.219630 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.219803 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.219726 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.219803 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.219760 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47sb5\" (UniqueName: \"kubernetes.io/projected/12920f25-3947-466c-8046-3bc0e4c9f458-kube-api-access-47sb5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.219891 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.219826 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.320531 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.320495 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.320531 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.320538 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47sb5\" (UniqueName: \"kubernetes.io/projected/12920f25-3947-466c-8046-3bc0e4c9f458-kube-api-access-47sb5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.320701 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.320574 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.320701 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.320590 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.320701 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.320618 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12920f25-3947-466c-8046-3bc0e4c9f458-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.320701 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.320653 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.321001 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.320977 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.321092 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.320987 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.321092 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.321046 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.322807 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.322781 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12920f25-3947-466c-8046-3bc0e4c9f458-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.323089 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.323055 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12920f25-3947-466c-8046-3bc0e4c9f458-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.332524 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.332504 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47sb5\" (UniqueName: \"kubernetes.io/projected/12920f25-3947-466c-8046-3bc0e4c9f458-kube-api-access-47sb5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld\" (UID: \"12920f25-3947-466c-8046-3bc0e4c9f458\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.490992 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.490975 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:46.630732 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.630704 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld"] Jul 15 10:29:46.632763 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:29:46.632740 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12920f25_3947_466c_8046_3bc0e4c9f458.slice/crio-7edbb0fd294b752525ea1ab108fcc7f53aeea3abd48f2f4dd3ca288f22ba3881 WatchSource:0}: Error finding container 7edbb0fd294b752525ea1ab108fcc7f53aeea3abd48f2f4dd3ca288f22ba3881: Status 404 returned error can't find the container with id 7edbb0fd294b752525ea1ab108fcc7f53aeea3abd48f2f4dd3ca288f22ba3881 Jul 15 10:29:46.936588 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:46.936499 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" event={"ID":"12920f25-3947-466c-8046-3bc0e4c9f458","Type":"ContainerStarted","Data":"7edbb0fd294b752525ea1ab108fcc7f53aeea3abd48f2f4dd3ca288f22ba3881"} Jul 15 10:29:47.939488 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:47.939454 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" event={"ID":"12920f25-3947-466c-8046-3bc0e4c9f458","Type":"ContainerStarted","Data":"73b9dc2acedcaae8a3f2f286326d73d8c11a007d63bb49923aced573bc6e379b"} Jul 15 10:29:47.959630 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:47.959578 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" podStartSLOduration=0.875178045 podStartE2EDuration="1.959562878s" totalImagesPullingTime="1.084384833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:29:47.959165541 +0000 UTC m=+591.475158027" watchObservedRunningTime="2026-07-15 10:29:47.959562878 +0000 UTC m=+591.475555363" Jul 15 10:29:56.491868 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:56.491829 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:56.492319 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:56.491903 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:56.503761 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:56.503737 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:56.991837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:56.991813 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld" Jul 15 10:29:57.661955 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:57.661930 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:29:57.662420 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:29:57.662203 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:30:14.685474 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.685434 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v"] Jul 15 10:30:14.690984 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.690964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.693568 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.693546 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 10:30:14.700632 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.700607 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v"] Jul 15 10:30:14.752401 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.752377 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35567337-2d5b-41ef-87cf-af5589350fe8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.752514 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.752415 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.752514 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.752435 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.752514 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.752459 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f95m\" (UniqueName: \"kubernetes.io/projected/35567337-2d5b-41ef-87cf-af5589350fe8-kube-api-access-4f95m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.752514 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.752492 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.752664 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.752539 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.853757 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.853724 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.853899 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.853809 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35567337-2d5b-41ef-87cf-af5589350fe8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.853899 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.853850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.853899 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.853880 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.854075 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.853904 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f95m\" (UniqueName: \"kubernetes.io/projected/35567337-2d5b-41ef-87cf-af5589350fe8-kube-api-access-4f95m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.854075 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.853932 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.854198 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.854157 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.854329 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.854311 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.854390 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.854335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.856036 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.856014 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35567337-2d5b-41ef-87cf-af5589350fe8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.856256 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.856237 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35567337-2d5b-41ef-87cf-af5589350fe8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:14.864052 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:14.864028 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f95m\" (UniqueName: \"kubernetes.io/projected/35567337-2d5b-41ef-87cf-af5589350fe8-kube-api-access-4f95m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gnt8v\" (UID: \"35567337-2d5b-41ef-87cf-af5589350fe8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:15.003895 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:15.003874 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:15.143373 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:15.143350 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v"] Jul 15 10:30:15.144764 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:30:15.144738 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35567337_2d5b_41ef_87cf_af5589350fe8.slice/crio-20675bfdd47fba25ad8e289f7f756f7431bb815d603ade52cdac64fc7cff88d6 WatchSource:0}: Error finding container 20675bfdd47fba25ad8e289f7f756f7431bb815d603ade52cdac64fc7cff88d6: Status 404 returned error can't find the container with id 20675bfdd47fba25ad8e289f7f756f7431bb815d603ade52cdac64fc7cff88d6 Jul 15 10:30:15.146099 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:15.146082 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 10:30:16.078488 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:16.078453 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" event={"ID":"35567337-2d5b-41ef-87cf-af5589350fe8","Type":"ContainerStarted","Data":"c3466c792ca60ed769685244fe14608c368d99b00ef2778a816563a79e264ca9"} Jul 15 10:30:16.078841 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:16.078516 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" event={"ID":"35567337-2d5b-41ef-87cf-af5589350fe8","Type":"ContainerStarted","Data":"20675bfdd47fba25ad8e289f7f756f7431bb815d603ade52cdac64fc7cff88d6"} Jul 15 10:30:16.101291 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:16.101247 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" podStartSLOduration=1.8075927809999999 podStartE2EDuration="2.101233655s" totalImagesPullingTime="293.640874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:30:16.099154664 +0000 UTC m=+619.615147150" watchObservedRunningTime="2026-07-15 10:30:16.101233655 +0000 UTC m=+619.617226139" Jul 15 10:30:25.004864 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.004780 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:25.004864 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.004831 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:25.017116 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.017093 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:25.141010 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.140972 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gnt8v" Jul 15 10:30:25.788118 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.788077 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g"] Jul 15 10:30:25.794803 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.794781 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:25.797698 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.797675 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 15 10:30:25.801850 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.801827 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g"] Jul 15 10:30:25.944719 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.944694 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:25.944878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.944726 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0c6e90-33a3-42d8-94bb-68d904950a71-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:25.944878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.944800 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkp4r\" (UniqueName: \"kubernetes.io/projected/8c0c6e90-33a3-42d8-94bb-68d904950a71-kube-api-access-dkp4r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:25.944878 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.944840 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:25.945044 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.944901 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:25.945044 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:25.944948 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.045827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.045757 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.045827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.045802 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.045827 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.045823 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0c6e90-33a3-42d8-94bb-68d904950a71-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.046306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.045870 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkp4r\" (UniqueName: \"kubernetes.io/projected/8c0c6e90-33a3-42d8-94bb-68d904950a71-kube-api-access-dkp4r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.046306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.045903 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.046306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.045955 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.046306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.046119 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.046306 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.046188 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.046516 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.046350 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.048154 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.048134 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c0c6e90-33a3-42d8-94bb-68d904950a71-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.048349 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.048332 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0c6e90-33a3-42d8-94bb-68d904950a71-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.056165 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.056142 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkp4r\" (UniqueName: \"kubernetes.io/projected/8c0c6e90-33a3-42d8-94bb-68d904950a71-kube-api-access-dkp4r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g\" (UID: \"8c0c6e90-33a3-42d8-94bb-68d904950a71\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.107198 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.107178 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:26.252864 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:26.252833 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g"] Jul 15 10:30:26.254490 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:30:26.254465 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0c6e90_33a3_42d8_94bb_68d904950a71.slice/crio-8e083d4fd3812879137060a68896b1aea5db267d19fb083dbe324be6662627ca WatchSource:0}: Error finding container 8e083d4fd3812879137060a68896b1aea5db267d19fb083dbe324be6662627ca: Status 404 returned error can't find the container with id 8e083d4fd3812879137060a68896b1aea5db267d19fb083dbe324be6662627ca Jul 15 10:30:27.142139 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:27.142103 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" event={"ID":"8c0c6e90-33a3-42d8-94bb-68d904950a71","Type":"ContainerStarted","Data":"116ce187a10552c28dcf1047903588e75fd3ea6017bb3eed857a43b3935769a0"} Jul 15 10:30:27.142509 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:27.142159 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" event={"ID":"8c0c6e90-33a3-42d8-94bb-68d904950a71","Type":"ContainerStarted","Data":"8e083d4fd3812879137060a68896b1aea5db267d19fb083dbe324be6662627ca"} Jul 15 10:30:27.164338 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:27.164289 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" podStartSLOduration=1.974617852 podStartE2EDuration="2.164275163s" totalImagesPullingTime="189.657311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:30:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:30:27.161609519 +0000 UTC m=+630.677602005" watchObservedRunningTime="2026-07-15 10:30:27.164275163 +0000 UTC m=+630.680267647" Jul 15 10:30:36.108253 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:36.108222 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:36.108629 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:36.108283 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:36.122113 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:36.122086 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:30:36.195194 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:30:36.195174 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g" Jul 15 10:33:18.368024 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.367984 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b5f89b76-l7zzc"] Jul 15 10:33:18.368626 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.368248 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" podUID="7ec89c51-2afd-4cdb-8ab2-eceb86083639" containerName="manager" containerID="cri-o://e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4" gracePeriod=30 Jul 15 10:33:18.620079 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.619997 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:33:18.751297 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.751270 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7ec89c51-2afd-4cdb-8ab2-eceb86083639-webhook-cert\") pod \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\" (UID: \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\") " Jul 15 10:33:18.751433 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.751329 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gm9r\" (UniqueName: \"kubernetes.io/projected/7ec89c51-2afd-4cdb-8ab2-eceb86083639-kube-api-access-2gm9r\") pod \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\" (UID: \"7ec89c51-2afd-4cdb-8ab2-eceb86083639\") " Jul 15 10:33:18.753241 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.753215 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ec89c51-2afd-4cdb-8ab2-eceb86083639-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7ec89c51-2afd-4cdb-8ab2-eceb86083639" (UID: "7ec89c51-2afd-4cdb-8ab2-eceb86083639"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:33:18.753353 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.753337 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ec89c51-2afd-4cdb-8ab2-eceb86083639-kube-api-access-2gm9r" (OuterVolumeSpecName: "kube-api-access-2gm9r") pod "7ec89c51-2afd-4cdb-8ab2-eceb86083639" (UID: "7ec89c51-2afd-4cdb-8ab2-eceb86083639"). InnerVolumeSpecName "kube-api-access-2gm9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:33:18.852219 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.852196 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7ec89c51-2afd-4cdb-8ab2-eceb86083639-webhook-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:33:18.852219 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:18.852217 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gm9r\" (UniqueName: \"kubernetes.io/projected/7ec89c51-2afd-4cdb-8ab2-eceb86083639-kube-api-access-2gm9r\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:33:19.035587 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.035547 2585 generic.go:356] "Generic (PLEG): container finished" podID="7ec89c51-2afd-4cdb-8ab2-eceb86083639" containerID="e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4" exitCode=0 Jul 15 10:33:19.035714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.035590 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" Jul 15 10:33:19.035714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.035628 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" event={"ID":"7ec89c51-2afd-4cdb-8ab2-eceb86083639","Type":"ContainerDied","Data":"e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4"} Jul 15 10:33:19.035714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.035683 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5f89b76-l7zzc" event={"ID":"7ec89c51-2afd-4cdb-8ab2-eceb86083639","Type":"ContainerDied","Data":"3d07df5f077a1fde5576108a1b18a4b38ee916d0a22d61a98b2e2c8d09572e86"} Jul 15 10:33:19.035714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.035705 2585 scope.go:117] "RemoveContainer" containerID="e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4" Jul 15 10:33:19.044432 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.044415 2585 scope.go:117] "RemoveContainer" containerID="e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4" Jul 15 10:33:19.044692 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:33:19.044671 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4\": container with ID starting with e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4 not found: ID does not exist" containerID="e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4" Jul 15 10:33:19.044739 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.044701 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4"} err="failed to get container status \"e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4\": rpc error: code = NotFound desc = could not find container \"e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4\": container with ID starting with e98fbdf4f790e24c53df17c7f2d520f9d923f9369aa9e67507e27ce8db64ead4 not found: ID does not exist" Jul 15 10:33:19.058211 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.058189 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b5f89b76-l7zzc"] Jul 15 10:33:19.062407 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.062388 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57b5f89b76-l7zzc"] Jul 15 10:33:19.843683 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.843652 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57b5f89b76-kjv6c"] Jul 15 10:33:19.844085 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.844058 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ec89c51-2afd-4cdb-8ab2-eceb86083639" containerName="manager" Jul 15 10:33:19.844085 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.844086 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ec89c51-2afd-4cdb-8ab2-eceb86083639" containerName="manager" Jul 15 10:33:19.844188 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.844178 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ec89c51-2afd-4cdb-8ab2-eceb86083639" containerName="manager" Jul 15 10:33:19.848410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.848395 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:19.854818 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.854793 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 10:33:19.854818 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.854805 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q6q65\"" Jul 15 10:33:19.855094 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.854852 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 10:33:19.862011 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.861986 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5f89b76-kjv6c"] Jul 15 10:33:19.961244 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.961219 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4j9q\" (UniqueName: \"kubernetes.io/projected/cbef26f1-56bf-441e-8702-1000f6a4ae50-kube-api-access-b4j9q\") pod \"maas-controller-57b5f89b76-kjv6c\" (UID: \"cbef26f1-56bf-441e-8702-1000f6a4ae50\") " pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:19.961367 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:19.961257 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbef26f1-56bf-441e-8702-1000f6a4ae50-webhook-cert\") pod \"maas-controller-57b5f89b76-kjv6c\" (UID: \"cbef26f1-56bf-441e-8702-1000f6a4ae50\") " pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:20.062513 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:20.062491 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4j9q\" (UniqueName: \"kubernetes.io/projected/cbef26f1-56bf-441e-8702-1000f6a4ae50-kube-api-access-b4j9q\") pod \"maas-controller-57b5f89b76-kjv6c\" (UID: \"cbef26f1-56bf-441e-8702-1000f6a4ae50\") " pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:20.062626 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:20.062528 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbef26f1-56bf-441e-8702-1000f6a4ae50-webhook-cert\") pod \"maas-controller-57b5f89b76-kjv6c\" (UID: \"cbef26f1-56bf-441e-8702-1000f6a4ae50\") " pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:20.064875 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:20.064855 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbef26f1-56bf-441e-8702-1000f6a4ae50-webhook-cert\") pod \"maas-controller-57b5f89b76-kjv6c\" (UID: \"cbef26f1-56bf-441e-8702-1000f6a4ae50\") " pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:20.078873 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:20.078850 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4j9q\" (UniqueName: \"kubernetes.io/projected/cbef26f1-56bf-441e-8702-1000f6a4ae50-kube-api-access-b4j9q\") pod \"maas-controller-57b5f89b76-kjv6c\" (UID: \"cbef26f1-56bf-441e-8702-1000f6a4ae50\") " pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:20.158056 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:20.158003 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:20.297030 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:20.296916 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5f89b76-kjv6c"] Jul 15 10:33:20.299484 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:33:20.299458 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbef26f1_56bf_441e_8702_1000f6a4ae50.slice/crio-b0b36d91ffd98d663277ead16484a3b98ccbd3aacae457ff1fa0681b96b17cb7 WatchSource:0}: Error finding container b0b36d91ffd98d663277ead16484a3b98ccbd3aacae457ff1fa0681b96b17cb7: Status 404 returned error can't find the container with id b0b36d91ffd98d663277ead16484a3b98ccbd3aacae457ff1fa0681b96b17cb7 Jul 15 10:33:20.934804 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:20.934778 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ec89c51-2afd-4cdb-8ab2-eceb86083639" path="/var/lib/kubelet/pods/7ec89c51-2afd-4cdb-8ab2-eceb86083639/volumes" Jul 15 10:33:21.047740 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:21.047707 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5f89b76-kjv6c" event={"ID":"cbef26f1-56bf-441e-8702-1000f6a4ae50","Type":"ContainerStarted","Data":"cbd83c0e11cd2164a551a8015c2f93a7641252455259bbd214787d3d3dcf90bf"} Jul 15 10:33:21.047894 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:21.047755 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5f89b76-kjv6c" event={"ID":"cbef26f1-56bf-441e-8702-1000f6a4ae50","Type":"ContainerStarted","Data":"b0b36d91ffd98d663277ead16484a3b98ccbd3aacae457ff1fa0681b96b17cb7"} Jul 15 10:33:21.047894 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:21.047776 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:33:21.068824 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:21.068780 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57b5f89b76-kjv6c" podStartSLOduration=1.4466091300000001 podStartE2EDuration="2.068765715s" totalImagesPullingTime="622.156585ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:33:21.06772992 +0000 UTC m=+804.583722406" watchObservedRunningTime="2026-07-15 10:33:21.068765715 +0000 UTC m=+804.584758234" Jul 15 10:33:32.055729 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:33:32.055696 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57b5f89b76-kjv6c" Jul 15 10:34:57.699578 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:34:57.699549 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:34:57.702098 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:34:57.702072 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:38:10.588660 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:10.588583 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c"] Jul 15 10:38:10.589178 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:10.588745 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" podUID="09f532d4-ae65-4ccd-a6c7-5d8860e84020" containerName="manager" containerID="cri-o://6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0" gracePeriod=10 Jul 15 10:38:12.743762 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:12.743739 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:38:12.872201 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:12.872136 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f532d4-ae65-4ccd-a6c7-5d8860e84020-extensions-socket-volume\") pod \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\" (UID: \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\") " Jul 15 10:38:12.872326 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:12.872225 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8l6p\" (UniqueName: \"kubernetes.io/projected/09f532d4-ae65-4ccd-a6c7-5d8860e84020-kube-api-access-l8l6p\") pod \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\" (UID: \"09f532d4-ae65-4ccd-a6c7-5d8860e84020\") " Jul 15 10:38:12.872492 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:12.872468 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f532d4-ae65-4ccd-a6c7-5d8860e84020-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "09f532d4-ae65-4ccd-a6c7-5d8860e84020" (UID: "09f532d4-ae65-4ccd-a6c7-5d8860e84020"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:38:12.874185 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:12.874164 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f532d4-ae65-4ccd-a6c7-5d8860e84020-kube-api-access-l8l6p" (OuterVolumeSpecName: "kube-api-access-l8l6p") pod "09f532d4-ae65-4ccd-a6c7-5d8860e84020" (UID: "09f532d4-ae65-4ccd-a6c7-5d8860e84020"). InnerVolumeSpecName "kube-api-access-l8l6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:38:12.972938 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:12.972914 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f532d4-ae65-4ccd-a6c7-5d8860e84020-extensions-socket-volume\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:38:12.973012 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:12.972939 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8l6p\" (UniqueName: \"kubernetes.io/projected/09f532d4-ae65-4ccd-a6c7-5d8860e84020-kube-api-access-l8l6p\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:38:13.544215 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.544181 2585 generic.go:356] "Generic (PLEG): container finished" podID="09f532d4-ae65-4ccd-a6c7-5d8860e84020" containerID="6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0" exitCode=0 Jul 15 10:38:13.544360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.544242 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" Jul 15 10:38:13.544360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.544246 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" event={"ID":"09f532d4-ae65-4ccd-a6c7-5d8860e84020","Type":"ContainerDied","Data":"6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0"} Jul 15 10:38:13.544360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.544325 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c" event={"ID":"09f532d4-ae65-4ccd-a6c7-5d8860e84020","Type":"ContainerDied","Data":"b3a13a6b423814e09197371e257cfff3b58b16e88a1f890f6c8fb115d60ec2b7"} Jul 15 10:38:13.544360 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.544353 2585 scope.go:117] "RemoveContainer" containerID="6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0" Jul 15 10:38:13.554219 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.554193 2585 scope.go:117] "RemoveContainer" containerID="6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0" Jul 15 10:38:13.554504 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:38:13.554481 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0\": container with ID starting with 6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0 not found: ID does not exist" containerID="6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0" Jul 15 10:38:13.554575 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.554513 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0"} err="failed to get container status \"6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0\": rpc error: code = NotFound desc = could not find container \"6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0\": container with ID starting with 6e5e71b0f34bc92912f4b9b8d1f1d8ce3dce2939723f4ffa863b7b0d74bcb0f0 not found: ID does not exist" Jul 15 10:38:13.572639 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.572618 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c"] Jul 15 10:38:13.576342 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:13.576318 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phm6c"] Jul 15 10:38:14.934402 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:38:14.934367 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09f532d4-ae65-4ccd-a6c7-5d8860e84020" path="/var/lib/kubelet/pods/09f532d4-ae65-4ccd-a6c7-5d8860e84020/volumes" Jul 15 10:39:11.729201 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.729160 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr"] Jul 15 10:39:11.729686 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.729581 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f532d4-ae65-4ccd-a6c7-5d8860e84020" containerName="manager" Jul 15 10:39:11.729686 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.729592 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f532d4-ae65-4ccd-a6c7-5d8860e84020" containerName="manager" Jul 15 10:39:11.729686 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.729652 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="09f532d4-ae65-4ccd-a6c7-5d8860e84020" containerName="manager" Jul 15 10:39:11.733863 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.733843 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:11.737111 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.737086 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gjg8k\"" Jul 15 10:39:11.747022 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.746999 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr"] Jul 15 10:39:11.851253 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.851227 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f9f276d-51a2-46a9-8bb3-f879c0ce7efd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tpfxr\" (UID: \"1f9f276d-51a2-46a9-8bb3-f879c0ce7efd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:11.851382 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.851297 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmszt\" (UniqueName: \"kubernetes.io/projected/1f9f276d-51a2-46a9-8bb3-f879c0ce7efd-kube-api-access-rmszt\") pod \"kuadrant-operator-controller-manager-86878c769f-tpfxr\" (UID: \"1f9f276d-51a2-46a9-8bb3-f879c0ce7efd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:11.952295 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.952269 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmszt\" (UniqueName: \"kubernetes.io/projected/1f9f276d-51a2-46a9-8bb3-f879c0ce7efd-kube-api-access-rmszt\") pod \"kuadrant-operator-controller-manager-86878c769f-tpfxr\" (UID: \"1f9f276d-51a2-46a9-8bb3-f879c0ce7efd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:11.952387 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.952348 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f9f276d-51a2-46a9-8bb3-f879c0ce7efd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tpfxr\" (UID: \"1f9f276d-51a2-46a9-8bb3-f879c0ce7efd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:11.952653 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.952637 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f9f276d-51a2-46a9-8bb3-f879c0ce7efd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tpfxr\" (UID: \"1f9f276d-51a2-46a9-8bb3-f879c0ce7efd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:11.965445 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:11.965422 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmszt\" (UniqueName: \"kubernetes.io/projected/1f9f276d-51a2-46a9-8bb3-f879c0ce7efd-kube-api-access-rmszt\") pod \"kuadrant-operator-controller-manager-86878c769f-tpfxr\" (UID: \"1f9f276d-51a2-46a9-8bb3-f879c0ce7efd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:12.044451 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:12.044432 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:12.193485 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:12.193407 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr"] Jul 15 10:39:12.196561 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:39:12.196534 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9f276d_51a2_46a9_8bb3_f879c0ce7efd.slice/crio-a1a79627431813848c090ba0ea1b2194a1f913906e42ed899a1ec96726e46782 WatchSource:0}: Error finding container a1a79627431813848c090ba0ea1b2194a1f913906e42ed899a1ec96726e46782: Status 404 returned error can't find the container with id a1a79627431813848c090ba0ea1b2194a1f913906e42ed899a1ec96726e46782 Jul 15 10:39:12.864460 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:12.864427 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" event={"ID":"1f9f276d-51a2-46a9-8bb3-f879c0ce7efd","Type":"ContainerStarted","Data":"10c6bb11e2e0f304deddf0fc68cb5a57d802d4c1064f7e17aa9341198f1fa14f"} Jul 15 10:39:12.864900 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:12.864482 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" event={"ID":"1f9f276d-51a2-46a9-8bb3-f879c0ce7efd","Type":"ContainerStarted","Data":"a1a79627431813848c090ba0ea1b2194a1f913906e42ed899a1ec96726e46782"} Jul 15 10:39:12.864900 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:12.864558 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:12.899502 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:12.899458 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" podStartSLOduration=1.899444323 podStartE2EDuration="1.899444323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:39:12.897679396 +0000 UTC m=+1156.413671882" watchObservedRunningTime="2026-07-15 10:39:12.899444323 +0000 UTC m=+1156.415436810" Jul 15 10:39:23.873191 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:23.873156 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tpfxr" Jul 15 10:39:57.733311 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:57.733282 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:39:57.736674 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:39:57.736650 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:44:57.765182 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:44:57.765154 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:44:57.770353 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:44:57.770333 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:46:28.795984 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.795947 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5"] Jul 15 10:46:28.800236 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.800212 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.803400 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.803021 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2b6d2c8d-gw-openshift-default-dockercfg-v498m\"" Jul 15 10:46:28.814400 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.814375 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5"] Jul 15 10:46:28.916566 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916535 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-socket\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.916714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916587 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-envoy\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.916714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916616 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnhpg\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-kube-api-access-lnhpg\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.916714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916642 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-data\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.916855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916721 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-credential-socket\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.916855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a39216-2a33-49ad-a45f-a5ca556186d5-istiod-ca-cert\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.916855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916814 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-certs\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.916855 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916837 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-podinfo\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:28.917028 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:28.916879 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-token\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018286 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018255 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-envoy\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018286 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018289 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnhpg\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-kube-api-access-lnhpg\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018504 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018306 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-data\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018504 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018347 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-credential-socket\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018504 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018381 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a39216-2a33-49ad-a45f-a5ca556186d5-istiod-ca-cert\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018504 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018410 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-certs\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018504 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018431 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-podinfo\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018504 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018468 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-token\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018822 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018549 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-socket\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.018822 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.018806 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-certs\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.019048 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.019023 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-socket\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.019219 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.019196 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-data\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.019304 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.019219 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a39216-2a33-49ad-a45f-a5ca556186d5-istiod-ca-cert\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.019304 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.019090 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-credential-socket\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.020775 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.020751 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-envoy\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.021002 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.020984 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-podinfo\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.032352 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.032328 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnhpg\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-kube-api-access-lnhpg\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.032627 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.032610 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-token\") pod \"e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.114195 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.114143 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:29.279371 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:46:29.279334 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a39216_2a33_49ad_a45f_a5ca556186d5.slice/crio-02ad9026c839135b2aac89badd69923bc395c1791ecafb4a1b3e846295bb5533 WatchSource:0}: Error finding container 02ad9026c839135b2aac89badd69923bc395c1791ecafb4a1b3e846295bb5533: Status 404 returned error can't find the container with id 02ad9026c839135b2aac89badd69923bc395c1791ecafb4a1b3e846295bb5533 Jul 15 10:46:29.281628 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.281584 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:46:29.281706 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.281671 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:46:29.281749 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.281713 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:46:29.281749 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:29.281730 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5"] Jul 15 10:46:30.104102 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:30.104015 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" event={"ID":"64a39216-2a33-49ad-a45f-a5ca556186d5","Type":"ContainerStarted","Data":"868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0"} Jul 15 10:46:30.104102 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:30.104104 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" event={"ID":"64a39216-2a33-49ad-a45f-a5ca556186d5","Type":"ContainerStarted","Data":"02ad9026c839135b2aac89badd69923bc395c1791ecafb4a1b3e846295bb5533"} Jul 15 10:46:30.114511 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:30.114485 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:30.119300 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:30.119279 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:30.139391 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:30.139350 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" podStartSLOduration=2.139337179 podStartE2EDuration="2.139337179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:46:30.137441097 +0000 UTC m=+1593.653433584" watchObservedRunningTime="2026-07-15 10:46:30.139337179 +0000 UTC m=+1593.655329663" Jul 15 10:46:31.108665 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:31.108633 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:31.109622 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:31.109605 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:40.188236 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:40.188158 2585 generic.go:356] "Generic (PLEG): container finished" podID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerID="868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0" exitCode=0 Jul 15 10:46:40.188236 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:40.188200 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" event={"ID":"64a39216-2a33-49ad-a45f-a5ca556186d5","Type":"ContainerDied","Data":"868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0"} Jul 15 10:46:40.188612 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:40.188563 2585 scope.go:117] "RemoveContainer" containerID="868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0" Jul 15 10:46:40.189558 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:40.189529 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:46:40.189645 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:40.189627 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:46:40.189699 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:40.189685 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 10:46:41.195471 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:41.195439 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" event={"ID":"64a39216-2a33-49ad-a45f-a5ca556186d5","Type":"ContainerStarted","Data":"2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d"} Jul 15 10:46:42.114626 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:42.114594 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:42.115714 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:42.115686 2585 patch_prober.go:28] interesting pod/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 10:46:42.115845 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:42.115734 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 10:46:43.114896 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:43.114865 2585 patch_prober.go:28] interesting pod/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 10:46:43.115281 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:43.114915 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 10:46:44.115034 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:44.115002 2585 patch_prober.go:28] interesting pod/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 10:46:44.115409 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:44.115078 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 10:46:45.118450 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:45.118420 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:45.118918 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:45.118800 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:46:45.119522 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:46:45.119501 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:47:34.659241 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:34.659207 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5"] Jul 15 10:47:34.659793 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:34.659438 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" containerID="cri-o://2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d" gracePeriod=30 Jul 15 10:47:39.915578 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.915551 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:47:39.932645 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932596 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-token\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.932645 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932629 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnhpg\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-kube-api-access-lnhpg\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.932645 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932649 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-credential-socket\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.932837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932667 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a39216-2a33-49ad-a45f-a5ca556186d5-istiod-ca-cert\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.932837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932723 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-socket\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.932837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932738 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-data\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.932837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932769 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-certs\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.932837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932792 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-envoy\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.932837 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932816 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-podinfo\") pod \"64a39216-2a33-49ad-a45f-a5ca556186d5\" (UID: \"64a39216-2a33-49ad-a45f-a5ca556186d5\") " Jul 15 10:47:39.933149 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.932927 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:47:39.933149 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.933033 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:47:39.933297 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.933276 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:39.933356 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.933300 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-credential-socket\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:39.933460 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.933437 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:47:39.933688 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.933661 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64a39216-2a33-49ad-a45f-a5ca556186d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 10:47:39.933842 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.933816 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:47:39.934817 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.934794 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 10:47:39.934817 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.934810 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:47:39.935056 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.935040 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-kube-api-access-lnhpg" (OuterVolumeSpecName: "kube-api-access-lnhpg") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "kube-api-access-lnhpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:47:39.935056 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:39.935114 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "64a39216-2a33-49ad-a45f-a5ca556186d5" (UID: "64a39216-2a33-49ad-a45f-a5ca556186d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:47:40.034710 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.034688 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-token\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:40.034710 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.034708 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnhpg\" (UniqueName: \"kubernetes.io/projected/64a39216-2a33-49ad-a45f-a5ca556186d5-kube-api-access-lnhpg\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:40.034858 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.034717 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a39216-2a33-49ad-a45f-a5ca556186d5-istiod-ca-cert\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:40.034858 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.034724 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-data\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:40.034858 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.034731 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-workload-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:40.034858 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.034739 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-envoy\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:40.034858 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.034746 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a39216-2a33-49ad-a45f-a5ca556186d5-istio-podinfo\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:47:40.502051 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.502018 2585 generic.go:356] "Generic (PLEG): container finished" podID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerID="2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d" exitCode=0 Jul 15 10:47:40.502227 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.502087 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" Jul 15 10:47:40.502227 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.502105 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" event={"ID":"64a39216-2a33-49ad-a45f-a5ca556186d5","Type":"ContainerDied","Data":"2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d"} Jul 15 10:47:40.502227 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.502161 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5" event={"ID":"64a39216-2a33-49ad-a45f-a5ca556186d5","Type":"ContainerDied","Data":"02ad9026c839135b2aac89badd69923bc395c1791ecafb4a1b3e846295bb5533"} Jul 15 10:47:40.502227 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.502185 2585 scope.go:117] "RemoveContainer" containerID="2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d" Jul 15 10:47:40.510990 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.510974 2585 scope.go:117] "RemoveContainer" containerID="868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0" Jul 15 10:47:40.519218 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.519198 2585 scope.go:117] "RemoveContainer" containerID="2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d" Jul 15 10:47:40.519472 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:47:40.519455 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d\": container with ID starting with 2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d not found: ID does not exist" containerID="2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d" Jul 15 10:47:40.519536 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.519480 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d"} err="failed to get container status \"2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d\": rpc error: code = NotFound desc = could not find container \"2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d\": container with ID starting with 2ad68e49f0afdd359628010f8f3aec22af8f92ac8005bf4c2f1a3557f32d108d not found: ID does not exist" Jul 15 10:47:40.519536 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.519496 2585 scope.go:117] "RemoveContainer" containerID="868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0" Jul 15 10:47:40.519733 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:47:40.519713 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0\": container with ID starting with 868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0 not found: ID does not exist" containerID="868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0" Jul 15 10:47:40.519795 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.519744 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0"} err="failed to get container status \"868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0\": rpc error: code = NotFound desc = could not find container \"868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0\": container with ID starting with 868f91964a4a0ac3cbb407f303020f0c4e1731f21a6928718e29245d1ffaf3c0 not found: ID does not exist" Jul 15 10:47:40.543745 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.543722 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5"] Jul 15 10:47:40.555738 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.555713 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2b6d2c8d-gw-openshift-default-b68b8c447-dnjc5"] Jul 15 10:47:40.937463 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:47:40.937391 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" path="/var/lib/kubelet/pods/64a39216-2a33-49ad-a45f-a5ca556186d5/volumes" Jul 15 10:49:57.800121 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:49:57.800091 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:49:57.804337 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:49:57.804313 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:54:57.833372 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:54:57.833342 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:54:57.838677 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:54:57.838654 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:57:25.487743 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.487669 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9"] Jul 15 10:57:25.488188 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.488117 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" Jul 15 10:57:25.488188 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.488131 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" Jul 15 10:57:25.488188 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.488140 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" Jul 15 10:57:25.488188 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.488148 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" Jul 15 10:57:25.488325 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.488242 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" Jul 15 10:57:25.488325 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.488251 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="64a39216-2a33-49ad-a45f-a5ca556186d5" containerName="istio-proxy" Jul 15 10:57:25.491419 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.491400 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.495410 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.495384 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a47e9a\"/\"default-dockercfg-hqmgx\"" Jul 15 10:57:25.495556 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.495437 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a47e9a\"/\"kube-root-ca.crt\"" Jul 15 10:57:25.495556 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.495437 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a47e9a\"/\"test-model-a47e9a-kserve-self-signed-certs\"" Jul 15 10:57:25.495718 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.495704 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a47e9a\"/\"openshift-service-ca.crt\"" Jul 15 10:57:25.505554 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.504037 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9"] Jul 15 10:57:25.551132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.551111 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrzk4\" (UniqueName: \"kubernetes.io/projected/6e33aa05-8535-4c0e-981b-a008d22f4502-kube-api-access-lrzk4\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.551266 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.551148 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-home\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.551266 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.551196 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-model-cache\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.551266 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.551247 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-dshm\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.551431 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.551272 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e33aa05-8535-4c0e-981b-a008d22f4502-tls-certs\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.551431 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.551354 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-tmp-dir\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.651779 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.651754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-model-cache\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.651916 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.651801 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-dshm\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.651916 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.651903 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e33aa05-8535-4c0e-981b-a008d22f4502-tls-certs\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.652029 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.651998 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-tmp-dir\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.652123 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.652035 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrzk4\" (UniqueName: \"kubernetes.io/projected/6e33aa05-8535-4c0e-981b-a008d22f4502-kube-api-access-lrzk4\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.652123 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.652115 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-home\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.652260 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.652235 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-model-cache\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.652387 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.652354 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-tmp-dir\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.652454 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.652415 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-home\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.653940 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.653921 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-dshm\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.654205 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.654190 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e33aa05-8535-4c0e-981b-a008d22f4502-tls-certs\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.662213 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.662191 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrzk4\" (UniqueName: \"kubernetes.io/projected/6e33aa05-8535-4c0e-981b-a008d22f4502-kube-api-access-lrzk4\") pod \"test-model-a47e9a-kserve-8688796467-w7pc9\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.802142 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.802058 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:25.946908 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:25.946885 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9"] Jul 15 10:57:25.948875 ip-10-0-130-114 kubenswrapper[2585]: W0715 10:57:25.948841 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e33aa05_8535_4c0e_981b_a008d22f4502.slice/crio-4e246ff1f3f86a83fac8bcaedd3ea0a6c174413bbf11cecdddb912d6f6eff039 WatchSource:0}: Error finding container 4e246ff1f3f86a83fac8bcaedd3ea0a6c174413bbf11cecdddb912d6f6eff039: Status 404 returned error can't find the container with id 4e246ff1f3f86a83fac8bcaedd3ea0a6c174413bbf11cecdddb912d6f6eff039 Jul 15 10:57:26.551600 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:26.551569 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" event={"ID":"6e33aa05-8535-4c0e-981b-a008d22f4502","Type":"ContainerStarted","Data":"b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470"} Jul 15 10:57:26.551946 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:26.551609 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" event={"ID":"6e33aa05-8535-4c0e-981b-a008d22f4502","Type":"ContainerStarted","Data":"4e246ff1f3f86a83fac8bcaedd3ea0a6c174413bbf11cecdddb912d6f6eff039"} Jul 15 10:57:26.571500 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:26.571458 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" podStartSLOduration=1.57144442 podStartE2EDuration="1.57144442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:57:26.569125013 +0000 UTC m=+2250.085117497" watchObservedRunningTime="2026-07-15 10:57:26.57144442 +0000 UTC m=+2250.087436904" Jul 15 10:57:35.802743 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:35.802705 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:35.803226 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:35.802757 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:35.815003 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:35.814980 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:36.613514 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:36.613476 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:57:46.069781 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:57:46.069743 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9"] Jul 15 10:58:01.089835 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.089774 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" podUID="6e33aa05-8535-4c0e-981b-a008d22f4502" containerName="main" containerID="cri-o://b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470" gracePeriod=45 Jul 15 10:58:01.339569 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.339543 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:58:01.348718 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.348666 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-dshm\") pod \"6e33aa05-8535-4c0e-981b-a008d22f4502\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " Jul 15 10:58:01.348718 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.348715 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-home\") pod \"6e33aa05-8535-4c0e-981b-a008d22f4502\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " Jul 15 10:58:01.348936 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.348781 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-tmp-dir\") pod \"6e33aa05-8535-4c0e-981b-a008d22f4502\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " Jul 15 10:58:01.348936 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.348823 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e33aa05-8535-4c0e-981b-a008d22f4502-tls-certs\") pod \"6e33aa05-8535-4c0e-981b-a008d22f4502\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " Jul 15 10:58:01.348936 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.348872 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrzk4\" (UniqueName: \"kubernetes.io/projected/6e33aa05-8535-4c0e-981b-a008d22f4502-kube-api-access-lrzk4\") pod \"6e33aa05-8535-4c0e-981b-a008d22f4502\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " Jul 15 10:58:01.348936 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.348894 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-model-cache\") pod \"6e33aa05-8535-4c0e-981b-a008d22f4502\" (UID: \"6e33aa05-8535-4c0e-981b-a008d22f4502\") " Jul 15 10:58:01.349172 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.349005 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-home" (OuterVolumeSpecName: "home") pod "6e33aa05-8535-4c0e-981b-a008d22f4502" (UID: "6e33aa05-8535-4c0e-981b-a008d22f4502"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:58:01.349172 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.349048 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6e33aa05-8535-4c0e-981b-a008d22f4502" (UID: "6e33aa05-8535-4c0e-981b-a008d22f4502"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:58:01.349278 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.349260 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-home\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:58:01.349348 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.349325 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-tmp-dir\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:58:01.349348 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.349260 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-model-cache" (OuterVolumeSpecName: "model-cache") pod "6e33aa05-8535-4c0e-981b-a008d22f4502" (UID: "6e33aa05-8535-4c0e-981b-a008d22f4502"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:58:01.351264 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.351241 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-dshm" (OuterVolumeSpecName: "dshm") pod "6e33aa05-8535-4c0e-981b-a008d22f4502" (UID: "6e33aa05-8535-4c0e-981b-a008d22f4502"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 10:58:01.351388 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.351364 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e33aa05-8535-4c0e-981b-a008d22f4502-kube-api-access-lrzk4" (OuterVolumeSpecName: "kube-api-access-lrzk4") pod "6e33aa05-8535-4c0e-981b-a008d22f4502" (UID: "6e33aa05-8535-4c0e-981b-a008d22f4502"). InnerVolumeSpecName "kube-api-access-lrzk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 10:58:01.351454 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.351351 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e33aa05-8535-4c0e-981b-a008d22f4502-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6e33aa05-8535-4c0e-981b-a008d22f4502" (UID: "6e33aa05-8535-4c0e-981b-a008d22f4502"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 10:58:01.450081 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.450032 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e33aa05-8535-4c0e-981b-a008d22f4502-tls-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:58:01.450081 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.450057 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrzk4\" (UniqueName: \"kubernetes.io/projected/6e33aa05-8535-4c0e-981b-a008d22f4502-kube-api-access-lrzk4\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:58:01.450235 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.450097 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-model-cache\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:58:01.450235 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.450109 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e33aa05-8535-4c0e-981b-a008d22f4502-dshm\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 10:58:01.731631 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.731563 2585 generic.go:356] "Generic (PLEG): container finished" podID="6e33aa05-8535-4c0e-981b-a008d22f4502" containerID="b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470" exitCode=0 Jul 15 10:58:01.731631 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.731608 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" Jul 15 10:58:01.731631 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.731617 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" event={"ID":"6e33aa05-8535-4c0e-981b-a008d22f4502","Type":"ContainerDied","Data":"b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470"} Jul 15 10:58:01.731816 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.731645 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9" event={"ID":"6e33aa05-8535-4c0e-981b-a008d22f4502","Type":"ContainerDied","Data":"4e246ff1f3f86a83fac8bcaedd3ea0a6c174413bbf11cecdddb912d6f6eff039"} Jul 15 10:58:01.731816 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.731661 2585 scope.go:117] "RemoveContainer" containerID="b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470" Jul 15 10:58:01.741253 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.741238 2585 scope.go:117] "RemoveContainer" containerID="b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470" Jul 15 10:58:01.741507 ip-10-0-130-114 kubenswrapper[2585]: E0715 10:58:01.741490 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470\": container with ID starting with b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470 not found: ID does not exist" containerID="b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470" Jul 15 10:58:01.741551 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.741515 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470"} err="failed to get container status \"b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470\": rpc error: code = NotFound desc = could not find container \"b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470\": container with ID starting with b14d620f0ba84531015a84843005c675be80c9e6ff0346ae83f4d6ceed9f0470 not found: ID does not exist" Jul 15 10:58:01.756580 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.756550 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9"] Jul 15 10:58:01.762132 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:01.762111 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a47e9a/test-model-a47e9a-kserve-8688796467-w7pc9"] Jul 15 10:58:02.934019 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:58:02.933984 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e33aa05-8535-4c0e-981b-a008d22f4502" path="/var/lib/kubelet/pods/6e33aa05-8535-4c0e-981b-a008d22f4502/volumes" Jul 15 10:59:57.867797 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:59:57.867770 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 10:59:57.875302 ip-10-0-130-114 kubenswrapper[2585]: I0715 10:59:57.875283 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 11:00:08.118246 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.118204 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj"] Jul 15 11:00:08.119288 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.119265 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e33aa05-8535-4c0e-981b-a008d22f4502" containerName="main" Jul 15 11:00:08.119416 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.119302 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e33aa05-8535-4c0e-981b-a008d22f4502" containerName="main" Jul 15 11:00:08.119481 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.119416 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e33aa05-8535-4c0e-981b-a008d22f4502" containerName="main" Jul 15 11:00:08.123108 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.123085 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.128176 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.128147 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zk7pv\"" Jul 15 11:00:08.128346 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.128329 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ceb9f0-serving-cert\"" Jul 15 11:00:08.141642 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.141619 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj"] Jul 15 11:00:08.223434 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.223409 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd93290-bdcd-4df4-b680-8e75ee174eaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.223555 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.223475 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q84vw\" (UniqueName: \"kubernetes.io/projected/6bd93290-bdcd-4df4-b680-8e75ee174eaf-kube-api-access-q84vw\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.223555 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.223511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.324848 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.324824 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd93290-bdcd-4df4-b680-8e75ee174eaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.324954 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.324868 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q84vw\" (UniqueName: \"kubernetes.io/projected/6bd93290-bdcd-4df4-b680-8e75ee174eaf-kube-api-access-q84vw\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.324954 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.324889 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.325041 ip-10-0-130-114 kubenswrapper[2585]: E0715 11:00:08.325028 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-serving-cert: secret "maas-api-e2e-shared-b-ceb9f0-serving-cert" not found Jul 15 11:00:08.325111 ip-10-0-130-114 kubenswrapper[2585]: E0715 11:00:08.325102 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls podName:6bd93290-bdcd-4df4-b680-8e75ee174eaf nodeName:}" failed. No retries permitted until 2026-07-15 11:00:08.825083785 +0000 UTC m=+2412.341076251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls") pod "maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" (UID: "6bd93290-bdcd-4df4-b680-8e75ee174eaf") : secret "maas-api-e2e-shared-b-ceb9f0-serving-cert" not found Jul 15 11:00:08.325631 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.325611 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd93290-bdcd-4df4-b680-8e75ee174eaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.345481 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.345457 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q84vw\" (UniqueName: \"kubernetes.io/projected/6bd93290-bdcd-4df4-b680-8e75ee174eaf-kube-api-access-q84vw\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.828728 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.828696 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:08.830866 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:08.830848 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:09.036690 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:09.036659 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:09.243000 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:09.242977 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj"] Jul 15 11:00:09.244601 ip-10-0-130-114 kubenswrapper[2585]: W0715 11:00:09.244571 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd93290_bdcd_4df4_b680_8e75ee174eaf.slice/crio-734d0e7e3149cf60b27ecfd37855781d13e45fc9914bb37b23ac711c95e8ec66 WatchSource:0}: Error finding container 734d0e7e3149cf60b27ecfd37855781d13e45fc9914bb37b23ac711c95e8ec66: Status 404 returned error can't find the container with id 734d0e7e3149cf60b27ecfd37855781d13e45fc9914bb37b23ac711c95e8ec66 Jul 15 11:00:09.246385 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:09.246358 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 11:00:09.375937 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:09.375859 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" event={"ID":"6bd93290-bdcd-4df4-b680-8e75ee174eaf","Type":"ContainerStarted","Data":"734d0e7e3149cf60b27ecfd37855781d13e45fc9914bb37b23ac711c95e8ec66"} Jul 15 11:00:10.381934 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:10.381901 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" event={"ID":"6bd93290-bdcd-4df4-b680-8e75ee174eaf","Type":"ContainerStarted","Data":"bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e"} Jul 15 11:00:10.382314 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:10.382011 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:00:10.417320 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:10.417267 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" podStartSLOduration=1.914087473 podStartE2EDuration="2.417249565s" totalImagesPullingTime="503.162092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:00:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:00:10.414524817 +0000 UTC m=+2413.930517314" watchObservedRunningTime="2026-07-15 11:00:10.417249565 +0000 UTC m=+2413.933242051" Jul 15 11:00:16.392549 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:00:16.392511 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:01:02.995446 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:02.995410 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc"] Jul 15 11:01:02.998928 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:02.998903 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.002850 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.002817 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ceb9f0\"/\"openshift-service-ca.crt\"" Jul 15 11:01:03.002967 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.002871 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ceb9f0\"/\"default-dockercfg-99mbq\"" Jul 15 11:01:03.002967 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.002891 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ceb9f0\"/\"kube-root-ca.crt\"" Jul 15 11:01:03.002967 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.002953 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ceb9f0\"/\"auth-test-model-ceb9f0-kserve-self-signed-certs\"" Jul 15 11:01:03.012700 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.012681 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc"] Jul 15 11:01:03.077783 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.077761 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tls-certs\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.077901 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.077791 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-dshm\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.077901 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.077809 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tmp-dir\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.077974 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.077898 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-model-cache\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.077974 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.077923 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-home\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.078043 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.077992 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db4jn\" (UniqueName: \"kubernetes.io/projected/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-kube-api-access-db4jn\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178395 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178368 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-dshm\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178395 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178397 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tmp-dir\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178551 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178436 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-model-cache\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178551 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178474 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-home\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178630 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178556 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db4jn\" (UniqueName: \"kubernetes.io/projected/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-kube-api-access-db4jn\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178630 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178614 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tls-certs\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178834 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178808 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tmp-dir\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178941 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178852 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-model-cache\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.178941 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.178875 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-home\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.180431 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.180413 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-dshm\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.180720 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.180703 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tls-certs\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.188709 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.188688 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db4jn\" (UniqueName: \"kubernetes.io/projected/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-kube-api-access-db4jn\") pod \"auth-test-model-ceb9f0-kserve-677c574d47-rpfxc\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.308202 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.308145 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:03.447078 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.447038 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc"] Jul 15 11:01:03.654773 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.654680 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" event={"ID":"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3","Type":"ContainerStarted","Data":"3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62"} Jul 15 11:01:03.654773 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.654724 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" event={"ID":"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3","Type":"ContainerStarted","Data":"31a64ef67d36e6838707a0c4efb55422137f78178e0be7044d204d90e26ddada"} Jul 15 11:01:03.675221 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:03.675169 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" podStartSLOduration=1.675154299 podStartE2EDuration="1.675154299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:01:03.672935673 +0000 UTC m=+2467.188928157" watchObservedRunningTime="2026-07-15 11:01:03.675154299 +0000 UTC m=+2467.191146783" Jul 15 11:01:13.308369 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:13.308338 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:13.308754 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:13.308382 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:13.320907 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:13.320879 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:01:13.714752 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:01:13.714683 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:03:04.526259 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:04.526228 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc"] Jul 15 11:03:19.545931 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.545838 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" podUID="379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" containerName="main" containerID="cri-o://3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62" gracePeriod=45 Jul 15 11:03:19.801175 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.801124 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:03:19.917909 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.917878 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-dshm\") pod \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " Jul 15 11:03:19.918055 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.917919 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-model-cache\") pod \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " Jul 15 11:03:19.918055 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.917949 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tmp-dir\") pod \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " Jul 15 11:03:19.918055 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.917987 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tls-certs\") pod \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " Jul 15 11:03:19.918055 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.918013 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db4jn\" (UniqueName: \"kubernetes.io/projected/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-kube-api-access-db4jn\") pod \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " Jul 15 11:03:19.918316 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.918193 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-home\") pod \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\" (UID: \"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3\") " Jul 15 11:03:19.918429 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.918400 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-model-cache" (OuterVolumeSpecName: "model-cache") pod "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" (UID: "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 11:03:19.918697 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.918439 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" (UID: "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 11:03:19.918913 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.918690 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-home" (OuterVolumeSpecName: "home") pod "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" (UID: "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 11:03:19.920496 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.920459 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" (UID: "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 11:03:19.920649 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.920628 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-dshm" (OuterVolumeSpecName: "dshm") pod "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" (UID: "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 11:03:19.920708 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:19.920647 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-kube-api-access-db4jn" (OuterVolumeSpecName: "kube-api-access-db4jn") pod "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" (UID: "379cdb30-3c9a-4d54-84b4-ede5c30c3cd3"). InnerVolumeSpecName "kube-api-access-db4jn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 11:03:20.019360 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.019333 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-dshm\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:03:20.019360 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.019357 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-model-cache\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:03:20.019497 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.019372 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tmp-dir\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:03:20.019497 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.019382 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-tls-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:03:20.019497 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.019393 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db4jn\" (UniqueName: \"kubernetes.io/projected/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-kube-api-access-db4jn\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:03:20.019497 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.019404 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3-home\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:03:20.371001 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.370971 2585 generic.go:356] "Generic (PLEG): container finished" podID="379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" containerID="3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62" exitCode=0 Jul 15 11:03:20.371186 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.371037 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" Jul 15 11:03:20.371186 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.371041 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" event={"ID":"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3","Type":"ContainerDied","Data":"3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62"} Jul 15 11:03:20.371186 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.371167 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc" event={"ID":"379cdb30-3c9a-4d54-84b4-ede5c30c3cd3","Type":"ContainerDied","Data":"31a64ef67d36e6838707a0c4efb55422137f78178e0be7044d204d90e26ddada"} Jul 15 11:03:20.371186 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.371184 2585 scope.go:117] "RemoveContainer" containerID="3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62" Jul 15 11:03:20.380577 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.380559 2585 scope.go:117] "RemoveContainer" containerID="3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62" Jul 15 11:03:20.380830 ip-10-0-130-114 kubenswrapper[2585]: E0715 11:03:20.380813 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62\": container with ID starting with 3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62 not found: ID does not exist" containerID="3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62" Jul 15 11:03:20.380876 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.380837 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62"} err="failed to get container status \"3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62\": rpc error: code = NotFound desc = could not find container \"3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62\": container with ID starting with 3b0303e23f2ca0566060efd9f283b516adde3d7343860702c930092ebcef2d62 not found: ID does not exist" Jul 15 11:03:20.395475 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.395454 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc"] Jul 15 11:03:20.399416 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.399396 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ceb9f0/auth-test-model-ceb9f0-kserve-677c574d47-rpfxc"] Jul 15 11:03:20.934313 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:03:20.934284 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" path="/var/lib/kubelet/pods/379cdb30-3c9a-4d54-84b4-ede5c30c3cd3/volumes" Jul 15 11:04:26.597138 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.597108 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck"] Jul 15 11:04:26.597576 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.597520 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" containerName="main" Jul 15 11:04:26.597576 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.597530 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" containerName="main" Jul 15 11:04:26.597660 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.597608 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="379cdb30-3c9a-4d54-84b4-ede5c30c3cd3" containerName="main" Jul 15 11:04:26.600768 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.600753 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.604433 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.604404 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b2a040\"/\"kube-root-ca.crt\"" Jul 15 11:04:26.604568 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.604482 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b2a040\"/\"openshift-service-ca.crt\"" Jul 15 11:04:26.604568 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.604549 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b2a040\"/\"rate-test-model-b2a040-kserve-self-signed-certs\"" Jul 15 11:04:26.604653 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.604624 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b2a040\"/\"default-dockercfg-c8298\"" Jul 15 11:04:26.612217 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.612196 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck"] Jul 15 11:04:26.662024 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.661994 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-tmp-dir\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.662167 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.662037 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-home\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.662167 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.662103 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc425cdd-8fbb-49d8-b489-cb153a26688b-tls-certs\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.662167 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.662145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-dshm\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.662273 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.662203 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpkwn\" (UniqueName: \"kubernetes.io/projected/bc425cdd-8fbb-49d8-b489-cb153a26688b-kube-api-access-zpkwn\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.662318 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.662272 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-model-cache\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.762935 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.762903 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpkwn\" (UniqueName: \"kubernetes.io/projected/bc425cdd-8fbb-49d8-b489-cb153a26688b-kube-api-access-zpkwn\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.763089 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.762979 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-model-cache\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.763089 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.763023 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-tmp-dir\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.763089 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.763056 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-home\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.763257 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.763123 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc425cdd-8fbb-49d8-b489-cb153a26688b-tls-certs\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.763257 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.763178 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-dshm\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.763443 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.763421 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-model-cache\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.763487 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.763447 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-home\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.763533 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.763506 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-tmp-dir\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.765404 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.765379 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-dshm\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.765662 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.765644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc425cdd-8fbb-49d8-b489-cb153a26688b-tls-certs\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.772193 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.772174 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpkwn\" (UniqueName: \"kubernetes.io/projected/bc425cdd-8fbb-49d8-b489-cb153a26688b-kube-api-access-zpkwn\") pod \"rate-test-model-b2a040-kserve-5f9cfb96b7-24tck\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:26.909941 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:26.909865 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:27.048637 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:27.048612 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck"] Jul 15 11:04:27.050457 ip-10-0-130-114 kubenswrapper[2585]: W0715 11:04:27.050428 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc425cdd_8fbb_49d8_b489_cb153a26688b.slice/crio-344366f5e0dec1ccc78f7c0dcd0939167e712de08ff39a8d5b2319bc2b58ba73 WatchSource:0}: Error finding container 344366f5e0dec1ccc78f7c0dcd0939167e712de08ff39a8d5b2319bc2b58ba73: Status 404 returned error can't find the container with id 344366f5e0dec1ccc78f7c0dcd0939167e712de08ff39a8d5b2319bc2b58ba73 Jul 15 11:04:27.732443 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:27.732407 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" event={"ID":"bc425cdd-8fbb-49d8-b489-cb153a26688b","Type":"ContainerStarted","Data":"017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774"} Jul 15 11:04:27.732816 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:27.732456 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" event={"ID":"bc425cdd-8fbb-49d8-b489-cb153a26688b","Type":"ContainerStarted","Data":"344366f5e0dec1ccc78f7c0dcd0939167e712de08ff39a8d5b2319bc2b58ba73"} Jul 15 11:04:27.753043 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:27.752996 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" podStartSLOduration=1.752981985 podStartE2EDuration="1.752981985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:04:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:04:27.752273348 +0000 UTC m=+2671.268265835" watchObservedRunningTime="2026-07-15 11:04:27.752981985 +0000 UTC m=+2671.268974466" Jul 15 11:04:36.910841 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:36.910808 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:36.911328 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:36.910855 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:36.923170 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:36.923148 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:37.795720 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:37.795690 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:04:57.900502 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:57.900422 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 11:04:57.908716 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:04:57.908696 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 11:05:31.739757 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:31.739722 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck"] Jul 15 11:05:46.759161 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:46.759124 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" podUID="bc425cdd-8fbb-49d8-b489-cb153a26688b" containerName="main" containerID="cri-o://017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774" gracePeriod=45 Jul 15 11:05:47.017877 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.017826 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:05:47.129002 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.128978 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-model-cache\") pod \"bc425cdd-8fbb-49d8-b489-cb153a26688b\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " Jul 15 11:05:47.129189 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129052 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc425cdd-8fbb-49d8-b489-cb153a26688b-tls-certs\") pod \"bc425cdd-8fbb-49d8-b489-cb153a26688b\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " Jul 15 11:05:47.129189 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129112 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpkwn\" (UniqueName: \"kubernetes.io/projected/bc425cdd-8fbb-49d8-b489-cb153a26688b-kube-api-access-zpkwn\") pod \"bc425cdd-8fbb-49d8-b489-cb153a26688b\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " Jul 15 11:05:47.129189 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129138 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-tmp-dir\") pod \"bc425cdd-8fbb-49d8-b489-cb153a26688b\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " Jul 15 11:05:47.129189 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129157 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-home\") pod \"bc425cdd-8fbb-49d8-b489-cb153a26688b\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " Jul 15 11:05:47.129189 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129173 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-dshm\") pod \"bc425cdd-8fbb-49d8-b489-cb153a26688b\" (UID: \"bc425cdd-8fbb-49d8-b489-cb153a26688b\") " Jul 15 11:05:47.129447 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129291 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-model-cache" (OuterVolumeSpecName: "model-cache") pod "bc425cdd-8fbb-49d8-b489-cb153a26688b" (UID: "bc425cdd-8fbb-49d8-b489-cb153a26688b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 11:05:47.129447 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129326 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bc425cdd-8fbb-49d8-b489-cb153a26688b" (UID: "bc425cdd-8fbb-49d8-b489-cb153a26688b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 11:05:47.129549 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129518 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-model-cache\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:05:47.129549 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129536 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-tmp-dir\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:05:47.129659 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.129542 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-home" (OuterVolumeSpecName: "home") pod "bc425cdd-8fbb-49d8-b489-cb153a26688b" (UID: "bc425cdd-8fbb-49d8-b489-cb153a26688b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 11:05:47.131198 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.131176 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-dshm" (OuterVolumeSpecName: "dshm") pod "bc425cdd-8fbb-49d8-b489-cb153a26688b" (UID: "bc425cdd-8fbb-49d8-b489-cb153a26688b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 11:05:47.131318 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.131206 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc425cdd-8fbb-49d8-b489-cb153a26688b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bc425cdd-8fbb-49d8-b489-cb153a26688b" (UID: "bc425cdd-8fbb-49d8-b489-cb153a26688b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 11:05:47.131318 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.131289 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc425cdd-8fbb-49d8-b489-cb153a26688b-kube-api-access-zpkwn" (OuterVolumeSpecName: "kube-api-access-zpkwn") pod "bc425cdd-8fbb-49d8-b489-cb153a26688b" (UID: "bc425cdd-8fbb-49d8-b489-cb153a26688b"). InnerVolumeSpecName "kube-api-access-zpkwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 11:05:47.147164 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.147142 2585 generic.go:356] "Generic (PLEG): container finished" podID="bc425cdd-8fbb-49d8-b489-cb153a26688b" containerID="017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774" exitCode=0 Jul 15 11:05:47.147262 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.147181 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" Jul 15 11:05:47.147262 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.147205 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" event={"ID":"bc425cdd-8fbb-49d8-b489-cb153a26688b","Type":"ContainerDied","Data":"017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774"} Jul 15 11:05:47.147262 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.147243 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck" event={"ID":"bc425cdd-8fbb-49d8-b489-cb153a26688b","Type":"ContainerDied","Data":"344366f5e0dec1ccc78f7c0dcd0939167e712de08ff39a8d5b2319bc2b58ba73"} Jul 15 11:05:47.147423 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.147277 2585 scope.go:117] "RemoveContainer" containerID="017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774" Jul 15 11:05:47.157529 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.157514 2585 scope.go:117] "RemoveContainer" containerID="017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774" Jul 15 11:05:47.157744 ip-10-0-130-114 kubenswrapper[2585]: E0715 11:05:47.157725 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774\": container with ID starting with 017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774 not found: ID does not exist" containerID="017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774" Jul 15 11:05:47.157796 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.157752 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774"} err="failed to get container status \"017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774\": rpc error: code = NotFound desc = could not find container \"017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774\": container with ID starting with 017e31013b88e912015ef813ad9580b9c5afaa5443a1b68592759e57f6651774 not found: ID does not exist" Jul 15 11:05:47.183601 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.183583 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck"] Jul 15 11:05:47.193612 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.193592 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b2a040/rate-test-model-b2a040-kserve-5f9cfb96b7-24tck"] Jul 15 11:05:47.230533 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.230512 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc425cdd-8fbb-49d8-b489-cb153a26688b-tls-certs\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:05:47.230533 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.230534 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpkwn\" (UniqueName: \"kubernetes.io/projected/bc425cdd-8fbb-49d8-b489-cb153a26688b-kube-api-access-zpkwn\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:05:47.230650 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.230543 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-home\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:05:47.230650 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:47.230551 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc425cdd-8fbb-49d8-b489-cb153a26688b-dshm\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:05:48.935503 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:05:48.935473 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc425cdd-8fbb-49d8-b489-cb153a26688b" path="/var/lib/kubelet/pods/bc425cdd-8fbb-49d8-b489-cb153a26688b/volumes" Jul 15 11:06:28.597285 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.597201 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj"] Jul 15 11:06:28.597776 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.597446 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" podUID="6bd93290-bdcd-4df4-b680-8e75ee174eaf" containerName="maas-api" containerID="cri-o://bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e" gracePeriod=30 Jul 15 11:06:28.852665 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.852612 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:06:28.868822 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.868797 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls\") pod \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " Jul 15 11:06:28.868927 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.868864 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q84vw\" (UniqueName: \"kubernetes.io/projected/6bd93290-bdcd-4df4-b680-8e75ee174eaf-kube-api-access-q84vw\") pod \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " Jul 15 11:06:28.868927 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.868926 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd93290-bdcd-4df4-b680-8e75ee174eaf-openshift-service-ca\") pod \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\" (UID: \"6bd93290-bdcd-4df4-b680-8e75ee174eaf\") " Jul 15 11:06:28.869295 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.869274 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bd93290-bdcd-4df4-b680-8e75ee174eaf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6bd93290-bdcd-4df4-b680-8e75ee174eaf" (UID: "6bd93290-bdcd-4df4-b680-8e75ee174eaf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 11:06:28.870964 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.870941 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd93290-bdcd-4df4-b680-8e75ee174eaf-kube-api-access-q84vw" (OuterVolumeSpecName: "kube-api-access-q84vw") pod "6bd93290-bdcd-4df4-b680-8e75ee174eaf" (UID: "6bd93290-bdcd-4df4-b680-8e75ee174eaf"). InnerVolumeSpecName "kube-api-access-q84vw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 11:06:28.871055 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.870973 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6bd93290-bdcd-4df4-b680-8e75ee174eaf" (UID: "6bd93290-bdcd-4df4-b680-8e75ee174eaf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 11:06:28.969798 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.969773 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q84vw\" (UniqueName: \"kubernetes.io/projected/6bd93290-bdcd-4df4-b680-8e75ee174eaf-kube-api-access-q84vw\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:06:28.969798 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.969797 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd93290-bdcd-4df4-b680-8e75ee174eaf-openshift-service-ca\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:06:28.969931 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:28.969806 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd93290-bdcd-4df4-b680-8e75ee174eaf-maas-api-tls\") on node \"ip-10-0-130-114.ec2.internal\" DevicePath \"\"" Jul 15 11:06:29.373952 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.373926 2585 generic.go:356] "Generic (PLEG): container finished" podID="6bd93290-bdcd-4df4-b680-8e75ee174eaf" containerID="bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e" exitCode=0 Jul 15 11:06:29.374103 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.373967 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" Jul 15 11:06:29.374103 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.374007 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" event={"ID":"6bd93290-bdcd-4df4-b680-8e75ee174eaf","Type":"ContainerDied","Data":"bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e"} Jul 15 11:06:29.374103 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.374045 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj" event={"ID":"6bd93290-bdcd-4df4-b680-8e75ee174eaf","Type":"ContainerDied","Data":"734d0e7e3149cf60b27ecfd37855781d13e45fc9914bb37b23ac711c95e8ec66"} Jul 15 11:06:29.374103 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.374084 2585 scope.go:117] "RemoveContainer" containerID="bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e" Jul 15 11:06:29.383076 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.383042 2585 scope.go:117] "RemoveContainer" containerID="bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e" Jul 15 11:06:29.383325 ip-10-0-130-114 kubenswrapper[2585]: E0715 11:06:29.383303 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e\": container with ID starting with bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e not found: ID does not exist" containerID="bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e" Jul 15 11:06:29.383387 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.383338 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e"} err="failed to get container status \"bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e\": rpc error: code = NotFound desc = could not find container \"bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e\": container with ID starting with bbe71356e9f06a6dd3a93161159cbf46dea71358041bddb1c7dc17c7ed36dc5e not found: ID does not exist" Jul 15 11:06:29.413208 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.413188 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj"] Jul 15 11:06:29.420664 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:29.420644 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ceb9f0-7774b86896-wp7cj"] Jul 15 11:06:30.934568 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:30.934540 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd93290-bdcd-4df4-b680-8e75ee174eaf" path="/var/lib/kubelet/pods/6bd93290-bdcd-4df4-b680-8e75ee174eaf/volumes" Jul 15 11:06:49.860981 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:49.860943 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vvpw6_bc6c6de1-4870-4ad0-963c-20fb21390c73/manager/0.log" Jul 15 11:06:49.978745 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:49.978708 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-95l2m_28ab1574-7658-4b02-8fff-8c3d11f2b1d9/manager/0.log" Jul 15 11:06:50.110452 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:50.110423 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57b5f89b76-kjv6c_cbef26f1-56bf-441e-8702-1000f6a4ae50/manager/0.log" Jul 15 11:06:50.764785 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:50.764758 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656947bb4-t6ttr_c7881688-dd73-4d4a-bafe-df3cd0219c78/copy-manifests/0.log" Jul 15 11:06:50.783935 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:50.783906 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656947bb4-t6ttr_c7881688-dd73-4d4a-bafe-df3cd0219c78/manager/0.log" Jul 15 11:06:51.171679 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:51.171616 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-86c7db645b-nlbvf_ab4b15a7-5e89-4ba5-8b70-cff7574ce483/maas-api/0.log" Jul 15 11:06:53.038155 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:53.038124 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-vgf9s_98356cee-41bc-488e-8459-4968c2a00d95/postgres/0.log" Jul 15 11:06:54.554964 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:54.554930 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lt79z_3490c1cd-5a43-4ac9-a4e2-27a75f05874d/manager/0.log" Jul 15 11:06:54.667832 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:54.667801 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kjd4r_1fe14792-29de-4773-866c-da83239e45ef/manager/0.log" Jul 15 11:06:54.776019 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:54.775995 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-44lpc_30ad177f-7132-4eae-bdc3-69dd28acf490/kuadrant-console-plugin/0.log" Jul 15 11:06:54.902623 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:54.902545 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-g6n4q_a91b7f6c-79b3-41e9-b724-9155a0ad9780/registry-server/0.log" Jul 15 11:06:55.024548 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:55.024530 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tpfxr_1f9f276d-51a2-46a9-8bb3-f879c0ce7efd/manager/0.log" Jul 15 11:06:55.278368 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:55.278344 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hdkqb_dbee81e3-1d56-4b0e-95e5-810243318d30/manager/0.log" Jul 15 11:06:55.988358 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:55.988325 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-544bf6b555-cc28c_4874dd31-0eaa-4dd8-bd1d-12451c4cea88/kube-auth-proxy/0.log" Jul 15 11:06:56.115382 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:56.115351 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7bdxq_8b1c4e79-589f-431c-96fb-63e32a17517b/istio-proxy/1.log" Jul 15 11:06:56.240695 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:56.240637 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-t9z66_5cada147-88b7-4860-bf21-49eb141c03fd/payload-pre-processing/0.log" Jul 15 11:06:56.367164 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:56.367137 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-mx5h8_919f6cd6-11e2-4e26-8246-51cea1e225d1/payload-processing/0.log" Jul 15 11:06:56.488527 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:56.488504 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6976c5c8-mk6gv_a2ccad75-c598-469a-adb8-10b2cb51c8e9/router/0.log" Jul 15 11:06:56.836286 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:56.836263 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dhnld_12920f25-3947-466c-8046-3bc0e4c9f458/main/0.log" Jul 15 11:06:57.064838 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:57.064816 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-gnt8v_35567337-2d5b-41ef-87cf-af5589350fe8/main/0.log" Jul 15 11:06:57.190610 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:06:57.190538 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhz82g_8c0c6e90-33a3-42d8-94bb-68d904950a71/main/0.log" Jul 15 11:07:06.279615 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:06.279583 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6z7nq_c0b6c7bc-5908-4525-aacc-0b8c3186c646/global-pull-secret-syncer/0.log" Jul 15 11:07:06.467194 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:06.467167 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rl85z_9188ae67-122d-4eae-aa25-98701dfe506b/konnectivity-agent/0.log" Jul 15 11:07:06.502195 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:06.502171 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-114.ec2.internal_b4114c320ef9f8839a18178df34d18c8/haproxy/0.log" Jul 15 11:07:12.260105 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:12.260076 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lt79z_3490c1cd-5a43-4ac9-a4e2-27a75f05874d/manager/0.log" Jul 15 11:07:12.303931 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:12.303909 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kjd4r_1fe14792-29de-4773-866c-da83239e45ef/manager/0.log" Jul 15 11:07:12.339139 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:12.339119 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-44lpc_30ad177f-7132-4eae-bdc3-69dd28acf490/kuadrant-console-plugin/0.log" Jul 15 11:07:12.392015 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:12.391988 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-g6n4q_a91b7f6c-79b3-41e9-b724-9155a0ad9780/registry-server/0.log" Jul 15 11:07:12.542912 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:12.542809 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tpfxr_1f9f276d-51a2-46a9-8bb3-f879c0ce7efd/manager/0.log" Jul 15 11:07:12.641630 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:12.641599 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hdkqb_dbee81e3-1d56-4b0e-95e5-810243318d30/manager/0.log" Jul 15 11:07:14.776563 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:14.776533 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-hszc4_b1483ec6-cc7d-4ab5-ae46-de2fa0f9a581/cluster-monitoring-operator/0.log" Jul 15 11:07:15.233126 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.233035 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gfkx2_1e32521d-cbf5-49f0-b1f3-7e74d215f5c0/node-exporter/0.log" Jul 15 11:07:15.260566 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.260541 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gfkx2_1e32521d-cbf5-49f0-b1f3-7e74d215f5c0/kube-rbac-proxy/0.log" Jul 15 11:07:15.289135 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.289115 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gfkx2_1e32521d-cbf5-49f0-b1f3-7e74d215f5c0/init-textfile/0.log" Jul 15 11:07:15.425908 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.425888 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd6b0204-7223-43e1-871c-eac61a998dc8/prometheus/0.log" Jul 15 11:07:15.456227 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.456205 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd6b0204-7223-43e1-871c-eac61a998dc8/config-reloader/0.log" Jul 15 11:07:15.487299 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.487280 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd6b0204-7223-43e1-871c-eac61a998dc8/thanos-sidecar/0.log" Jul 15 11:07:15.521287 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.521270 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd6b0204-7223-43e1-871c-eac61a998dc8/kube-rbac-proxy-web/0.log" Jul 15 11:07:15.553540 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.553521 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd6b0204-7223-43e1-871c-eac61a998dc8/kube-rbac-proxy/0.log" Jul 15 11:07:15.584169 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.584155 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd6b0204-7223-43e1-871c-eac61a998dc8/kube-rbac-proxy-thanos/0.log" Jul 15 11:07:15.617067 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.617048 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd6b0204-7223-43e1-871c-eac61a998dc8/init-config-reloader/0.log" Jul 15 11:07:15.652683 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.652663 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rk8np_69f3869f-2528-4450-96c9-f3d4f3b2ae2c/prometheus-operator/0.log" Jul 15 11:07:15.684811 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.684790 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rk8np_69f3869f-2528-4450-96c9-f3d4f3b2ae2c/kube-rbac-proxy/0.log" Jul 15 11:07:15.755881 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.755833 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bd7dd4f65-x77kq_3ac9e27e-5c04-4865-8a90-d6519decb39e/telemeter-client/0.log" Jul 15 11:07:15.783618 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.783600 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bd7dd4f65-x77kq_3ac9e27e-5c04-4865-8a90-d6519decb39e/reload/0.log" Jul 15 11:07:15.814112 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.814092 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bd7dd4f65-x77kq_3ac9e27e-5c04-4865-8a90-d6519decb39e/kube-rbac-proxy/0.log" Jul 15 11:07:15.860520 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.860501 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6"] Jul 15 11:07:15.860916 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.860904 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd93290-bdcd-4df4-b680-8e75ee174eaf" containerName="maas-api" Jul 15 11:07:15.860971 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.860918 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd93290-bdcd-4df4-b680-8e75ee174eaf" containerName="maas-api" Jul 15 11:07:15.860971 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.860932 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc425cdd-8fbb-49d8-b489-cb153a26688b" containerName="main" Jul 15 11:07:15.860971 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.860938 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc425cdd-8fbb-49d8-b489-cb153a26688b" containerName="main" Jul 15 11:07:15.861081 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.861007 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd93290-bdcd-4df4-b680-8e75ee174eaf" containerName="maas-api" Jul 15 11:07:15.861081 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.861018 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc425cdd-8fbb-49d8-b489-cb153a26688b" containerName="main" Jul 15 11:07:15.864106 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.864089 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:15.867055 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.867036 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-68cgz\"/\"kube-root-ca.crt\"" Jul 15 11:07:15.867317 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.867303 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-68cgz\"/\"openshift-service-ca.crt\"" Jul 15 11:07:15.867984 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.867966 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-68cgz\"/\"default-dockercfg-8vwgn\"" Jul 15 11:07:15.877843 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.877825 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6"] Jul 15 11:07:15.952423 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.952401 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-proc\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:15.952526 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.952443 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-podres\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:15.952526 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.952467 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2frl\" (UniqueName: \"kubernetes.io/projected/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-kube-api-access-c2frl\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:15.952645 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.952575 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-lib-modules\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:15.952645 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:15.952608 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-sys\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.053903 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.053848 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-proc\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.053903 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.053891 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-podres\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.054023 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.053913 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2frl\" (UniqueName: \"kubernetes.io/projected/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-kube-api-access-c2frl\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.054023 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.053961 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-proc\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.054130 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.054035 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-lib-modules\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.054130 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.054036 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-podres\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.054130 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.054056 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-sys\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.054130 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.054112 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-sys\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.054256 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.054222 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-lib-modules\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.064564 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.064541 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2frl\" (UniqueName: \"kubernetes.io/projected/6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6-kube-api-access-c2frl\") pod \"perf-node-gather-daemonset-mdkk6\" (UID: \"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6\") " pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.173717 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.173691 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.307583 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.307496 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6"] Jul 15 11:07:16.310100 ip-10-0-130-114 kubenswrapper[2585]: W0715 11:07:16.310045 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6c74c69b_5b7d_49d0_8e83_7d17edc3cfd6.slice/crio-872a6aa16d93f8603367786d17a392418f33f925b91bf977be970b3d3b3cabbf WatchSource:0}: Error finding container 872a6aa16d93f8603367786d17a392418f33f925b91bf977be970b3d3b3cabbf: Status 404 returned error can't find the container with id 872a6aa16d93f8603367786d17a392418f33f925b91bf977be970b3d3b3cabbf Jul 15 11:07:16.638100 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.638007 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" event={"ID":"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6","Type":"ContainerStarted","Data":"cdd25b181be29ba37123cbacbc217f42923dba38a0bae6bcd4e5ad53a7ad4f10"} Jul 15 11:07:16.638100 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.638086 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" event={"ID":"6c74c69b-5b7d-49d0-8e83-7d17edc3cfd6","Type":"ContainerStarted","Data":"872a6aa16d93f8603367786d17a392418f33f925b91bf977be970b3d3b3cabbf"} Jul 15 11:07:16.638258 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.638193 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:16.658652 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:16.658605 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" podStartSLOduration=1.658587993 podStartE2EDuration="1.658587993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:07:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:07:16.658150153 +0000 UTC m=+2840.174142639" watchObservedRunningTime="2026-07-15 11:07:16.658587993 +0000 UTC m=+2840.174580479" Jul 15 11:07:17.749952 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:17.749901 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/1.log" Jul 15 11:07:17.754369 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:17.754344 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bcmt7_7421f994-e37a-4cf5-830c-70a5fabc555d/console-operator/2.log" Jul 15 11:07:18.260280 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:18.260254 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78945ccfbf-pgd25_ce4a8c65-4ed4-4de5-8f3b-2f8877d285ea/console/0.log" Jul 15 11:07:18.907316 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:18.907272 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-9mmjb_b0621a97-3a04-46fd-af62-3a2af5065073/volume-data-source-validator/0.log" Jul 15 11:07:20.172805 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:20.172772 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kxkx5_ae01fde5-5868-4c83-9b4d-9b8e52cfd453/dns/0.log" Jul 15 11:07:20.200342 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:20.200313 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kxkx5_ae01fde5-5868-4c83-9b4d-9b8e52cfd453/kube-rbac-proxy/0.log" Jul 15 11:07:20.228948 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:20.228928 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-92kbz_d842f042-1ae5-4efe-a17a-7da9e8f76567/dns-node-resolver/0.log" Jul 15 11:07:20.901205 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:20.901182 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j4p58_dd8b7c91-b49c-45bb-ae5b-e5ebe251de69/node-ca/0.log" Jul 15 11:07:22.197336 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:22.197295 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-544bf6b555-cc28c_4874dd31-0eaa-4dd8-bd1d-12451c4cea88/kube-auth-proxy/0.log" Jul 15 11:07:22.240387 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:22.240360 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7bdxq_8b1c4e79-589f-431c-96fb-63e32a17517b/istio-proxy/0.log" Jul 15 11:07:22.270839 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:22.270813 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7bdxq_8b1c4e79-589f-431c-96fb-63e32a17517b/istio-proxy/1.log" Jul 15 11:07:22.302579 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:22.302547 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-t9z66_5cada147-88b7-4860-bf21-49eb141c03fd/payload-pre-processing/0.log" Jul 15 11:07:22.332556 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:22.332537 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-mx5h8_919f6cd6-11e2-4e26-8246-51cea1e225d1/payload-processing/0.log" Jul 15 11:07:22.364161 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:22.364139 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6976c5c8-mk6gv_a2ccad75-c598-469a-adb8-10b2cb51c8e9/router/0.log" Jul 15 11:07:22.653726 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:22.653705 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-68cgz/perf-node-gather-daemonset-mdkk6" Jul 15 11:07:23.211537 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:23.211512 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7mwj_e0e76093-626a-4493-a99e-e4ae77681681/serve-healthcheck-canary/0.log" Jul 15 11:07:23.726045 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:23.726021 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-6nkd9_5d2eab90-c6a8-4554-b303-102ede8d8570/insights-operator/0.log" Jul 15 11:07:23.726747 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:23.726732 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-6nkd9_5d2eab90-c6a8-4554-b303-102ede8d8570/insights-operator/1.log" Jul 15 11:07:23.752662 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:23.752642 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hzjrv_37035e8a-7489-4a0a-97fc-dbb3739cc852/kube-rbac-proxy/0.log" Jul 15 11:07:23.779732 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:23.779704 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hzjrv_37035e8a-7489-4a0a-97fc-dbb3739cc852/exporter/0.log" Jul 15 11:07:23.814423 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:23.814404 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hzjrv_37035e8a-7489-4a0a-97fc-dbb3739cc852/extractor/0.log" Jul 15 11:07:26.297828 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.297803 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-544bf6b555-cc28c_4874dd31-0eaa-4dd8-bd1d-12451c4cea88/kube-auth-proxy/0.log" Jul 15 11:07:26.326485 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.326462 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7bdxq_8b1c4e79-589f-431c-96fb-63e32a17517b/istio-proxy/0.log" Jul 15 11:07:26.356025 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.356007 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7bdxq_8b1c4e79-589f-431c-96fb-63e32a17517b/istio-proxy/1.log" Jul 15 11:07:26.375743 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.375708 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-t9z66_5cada147-88b7-4860-bf21-49eb141c03fd/payload-pre-processing/0.log" Jul 15 11:07:26.398634 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.398614 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-mx5h8_919f6cd6-11e2-4e26-8246-51cea1e225d1/payload-processing/0.log" Jul 15 11:07:26.416821 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.416803 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6976c5c8-mk6gv_a2ccad75-c598-469a-adb8-10b2cb51c8e9/router/0.log" Jul 15 11:07:26.490242 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.490217 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vvpw6_bc6c6de1-4870-4ad0-963c-20fb21390c73/manager/0.log" Jul 15 11:07:26.638820 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.638785 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-95l2m_28ab1574-7658-4b02-8fff-8c3d11f2b1d9/manager/0.log" Jul 15 11:07:26.730866 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:26.730841 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57b5f89b76-kjv6c_cbef26f1-56bf-441e-8702-1000f6a4ae50/manager/0.log" Jul 15 11:07:27.055693 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:27.055624 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656947bb4-t6ttr_c7881688-dd73-4d4a-bafe-df3cd0219c78/manager/0.log" Jul 15 11:07:27.075692 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:27.075672 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656947bb4-t6ttr_c7881688-dd73-4d4a-bafe-df3cd0219c78/copy-manifests/0.log" Jul 15 11:07:27.875562 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:27.875542 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-544bf6b555-cc28c_4874dd31-0eaa-4dd8-bd1d-12451c4cea88/kube-auth-proxy/0.log" Jul 15 11:07:27.900157 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:27.900128 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7bdxq_8b1c4e79-589f-431c-96fb-63e32a17517b/istio-proxy/0.log" Jul 15 11:07:27.926346 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:27.926322 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7bdxq_8b1c4e79-589f-431c-96fb-63e32a17517b/istio-proxy/1.log" Jul 15 11:07:27.946000 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:27.945981 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-t9z66_5cada147-88b7-4860-bf21-49eb141c03fd/payload-pre-processing/0.log" Jul 15 11:07:27.965703 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:27.965683 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-mx5h8_919f6cd6-11e2-4e26-8246-51cea1e225d1/payload-processing/0.log" Jul 15 11:07:27.980344 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:27.980328 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6976c5c8-mk6gv_a2ccad75-c598-469a-adb8-10b2cb51c8e9/router/0.log" Jul 15 11:07:33.920035 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:33.919978 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-t4dcj_481a73d5-b917-4f18-84eb-03aa60dc1e1a/kube-storage-version-migrator-operator/1.log" Jul 15 11:07:33.920998 ip-10-0-130-114 kubenswrapper[2585]: I0715 11:07:33.920975 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-t4dcj_481a73d5-b917-4f18-84eb-03aa60dc1e1a/kube-storage-version-migrator-operator/0.log"