Jul 27 14:16:56.523934 ip-10-0-133-223 systemd[1]: Starting Kubernetes Kubelet... Jul 27 14:16:56.898039 ip-10-0-133-223 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 14:16:56.898039 ip-10-0-133-223 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 27 14:16:56.898039 ip-10-0-133-223 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 14:16:56.898039 ip-10-0-133-223 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 27 14:16:56.898039 ip-10-0-133-223 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 14:16:56.898847 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.898731 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 27 14:16:56.903219 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903203 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903222 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903226 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903229 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903232 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903235 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903238 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903240 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903243 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903246 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903249 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903252 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903255 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903257 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903260 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903263 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903265 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903270 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903274 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 14:16:56.903341 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903277 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903280 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903283 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903286 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903289 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903292 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903298 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903301 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903303 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903306 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903309 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903311 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903314 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903316 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903320 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903323 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903326 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903329 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903331 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903334 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 14:16:56.903834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903337 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903340 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903342 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903345 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903348 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903352 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903355 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903357 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903360 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903363 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903365 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903368 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903370 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903373 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903375 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903378 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903381 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903385 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903387 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 14:16:56.904331 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903390 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903393 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903395 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903398 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903400 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903403 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903405 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903408 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903411 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903414 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903416 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903419 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903422 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903425 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903427 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903430 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903433 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903436 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903439 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903442 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 14:16:56.904851 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903444 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903447 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903449 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903452 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903454 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903457 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903464 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903469 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903473 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903478 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903482 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903485 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903488 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903491 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903493 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903502 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903506 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903508 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903511 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903516 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 14:16:56.905336 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.903521 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904263 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904270 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904273 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904277 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904280 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904282 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904285 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904288 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904291 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904293 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904296 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904299 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904301 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904304 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904306 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904309 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904312 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904316 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 14:16:56.905849 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904320 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904323 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904326 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904329 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904332 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904335 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904338 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904341 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904344 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904346 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904349 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904351 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904354 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904357 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904360 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904364 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904367 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904370 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904373 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904375 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 14:16:56.906333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904378 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904381 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904383 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904386 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904388 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904391 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904393 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904396 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904399 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904401 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904403 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904407 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904409 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904412 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904415 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904417 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904420 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904422 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904427 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 14:16:56.906847 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904430 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904433 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904436 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904439 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904441 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904444 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904447 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904450 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904453 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904456 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904458 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904461 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904464 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904467 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904470 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904472 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904475 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904477 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904480 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 14:16:56.907338 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904482 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904485 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904487 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904490 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904492 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904495 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904497 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904500 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904503 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904506 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904508 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904511 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904514 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904516 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904519 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904521 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904524 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904526 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904529 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904532 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 14:16:56.907835 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904535 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904538 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904540 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904928 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904934 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904937 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904940 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904943 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904946 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904949 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904951 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904954 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904956 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904959 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904962 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904964 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904967 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904970 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904972 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904974 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 14:16:56.908333 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904983 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904986 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904989 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904992 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904995 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.904997 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905000 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905003 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905005 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905008 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905011 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905013 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905016 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905019 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905022 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905025 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905027 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905030 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905033 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905036 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 14:16:56.908841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905038 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905040 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905043 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905046 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905048 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905050 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905053 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905055 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905058 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905060 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905063 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905065 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905067 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905077 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905081 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905084 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905087 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905090 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905093 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 14:16:56.909327 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905096 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905098 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905101 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905103 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905107 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905109 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905112 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905114 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905117 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905119 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905121 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905124 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905126 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905129 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905132 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905134 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905137 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905139 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905143 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 14:16:56.909834 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905146 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905149 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905151 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905154 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905156 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905159 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905161 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905164 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905176 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905179 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905182 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905185 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905187 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905190 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905193 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905195 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905198 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905200 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905202 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905205 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 14:16:56.910299 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905207 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905210 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905213 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905215 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905288 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905294 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905301 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905308 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905322 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905327 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905333 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905337 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905341 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905344 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905348 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905351 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905355 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905357 2555 flags.go:64] FLAG: --cgroup-root="" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905360 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905363 2555 flags.go:64] FLAG: --client-ca-file="" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905366 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905370 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905374 2555 flags.go:64] FLAG: --cluster-domain="" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905377 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 27 14:16:56.910793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905383 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905386 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905390 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905393 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905396 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905400 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905403 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905406 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905409 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905412 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905415 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905418 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905422 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905425 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905428 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905431 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905434 2555 flags.go:64] FLAG: --enable-server="true" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905437 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905441 2555 flags.go:64] FLAG: --event-burst="100" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905444 2555 flags.go:64] FLAG: --event-qps="50" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905447 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905450 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905455 2555 flags.go:64] FLAG: --eviction-hard="" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905459 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 27 14:16:56.911358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905462 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905465 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905467 2555 flags.go:64] FLAG: --eviction-soft="" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905470 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905473 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905476 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905483 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905486 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905490 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905493 2555 flags.go:64] FLAG: --feature-gates="" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905497 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905500 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905503 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905506 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905509 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905512 2555 flags.go:64] FLAG: --help="false" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905515 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-133-223.ec2.internal" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905518 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905521 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905524 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905527 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905531 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905533 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 27 14:16:56.911977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905536 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905539 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905542 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905545 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905548 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905551 2555 flags.go:64] FLAG: --kube-reserved="" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905554 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905558 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905561 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905564 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905567 2555 flags.go:64] FLAG: --lock-file="" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905569 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905572 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905575 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905581 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905584 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905592 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905596 2555 flags.go:64] FLAG: --logging-format="text" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905599 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905603 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905606 2555 flags.go:64] FLAG: --manifest-url="" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905608 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905613 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905616 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905620 2555 flags.go:64] FLAG: --max-pods="110" Jul 27 14:16:56.912551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905623 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905626 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905629 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905632 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905635 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905638 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905641 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905648 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905651 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905665 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905669 2555 flags.go:64] FLAG: --pod-cidr="" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905672 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905678 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905681 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905685 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905688 2555 flags.go:64] FLAG: --port="10250" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905691 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905694 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e5145e8e01c78a8d" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905697 2555 flags.go:64] FLAG: --qos-reserved="" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905700 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905703 2555 flags.go:64] FLAG: --register-node="true" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905706 2555 flags.go:64] FLAG: --register-with-taints="" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905710 2555 flags.go:64] FLAG: --registry-burst="10" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905713 2555 flags.go:64] FLAG: --registry-qps="5" Jul 27 14:16:56.913229 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905726 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905731 2555 flags.go:64] FLAG: --reserved-memory="" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905735 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905738 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905741 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905743 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905746 2555 flags.go:64] FLAG: --runonce="false" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905749 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905752 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905755 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905758 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905761 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905764 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905767 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905770 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905773 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905776 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905779 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905782 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905785 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905788 2555 flags.go:64] FLAG: --system-cgroups="" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905800 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905807 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905810 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905813 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 27 14:16:56.913837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905820 2555 flags.go:64] FLAG: --tls-min-version="" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905823 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905825 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905828 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905831 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905834 2555 flags.go:64] FLAG: --v="2" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905838 2555 flags.go:64] FLAG: --version="false" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905842 2555 flags.go:64] FLAG: --vmodule="" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905853 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.905857 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905972 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905976 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905979 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905982 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905985 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905988 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905990 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905993 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905996 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.905999 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906001 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906004 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 14:16:56.914448 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906009 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906013 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906017 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906020 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906022 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906026 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906030 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906035 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906038 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906041 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906044 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906046 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906049 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906052 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906055 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906057 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906060 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906062 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906066 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 14:16:56.915004 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906069 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906072 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906077 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906080 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906083 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906086 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906089 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906092 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906095 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906098 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906101 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906104 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906107 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906109 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906112 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906115 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906117 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906120 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906123 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906126 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 14:16:56.915463 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906130 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906133 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906135 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906138 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906140 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906143 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906146 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906148 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906151 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906153 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906156 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906160 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906163 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906167 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906170 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906172 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906175 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906178 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906180 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906183 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 14:16:56.916028 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906185 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906188 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906190 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906193 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906195 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906198 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906200 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906203 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906205 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906208 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906211 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906214 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906218 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906220 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906223 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906225 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906228 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906231 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906233 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 14:16:56.916520 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906236 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906238 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906241 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906243 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906247 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906250 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906252 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906256 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.906258 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.906267 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.913622 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.913638 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.914849 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.914858 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914916 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 14:16:56.916994 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914921 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914924 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914928 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914931 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914935 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914938 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914940 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914943 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914946 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914949 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914952 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914954 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914957 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914959 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914962 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914964 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914967 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914969 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914972 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914975 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 14:16:56.917384 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914977 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914980 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914982 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914985 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914988 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914990 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914993 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.914995 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915000 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915002 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915007 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915011 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915014 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915017 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915019 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915022 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915024 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915027 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915030 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 14:16:56.917864 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915034 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915037 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915040 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915043 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915045 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915048 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915050 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915053 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915055 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915058 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915061 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915063 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915065 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915068 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915071 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915073 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915076 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915078 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915081 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915083 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 14:16:56.918321 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915086 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915089 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915092 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915095 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915098 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915100 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915103 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915106 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915108 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915111 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915113 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915116 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915118 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915120 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915123 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915125 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915129 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915132 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915135 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 14:16:56.918841 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915137 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915140 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915142 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915145 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915147 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915150 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915152 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915154 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915157 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915160 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915162 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915164 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915167 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915170 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915173 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915175 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915179 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915181 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915184 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 14:16:56.919301 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915186 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.915191 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915300 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915305 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915308 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915312 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915316 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915319 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915322 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915324 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915327 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915330 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915333 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915335 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915338 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 14:16:56.919787 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915341 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915344 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915346 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915349 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915351 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915354 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915357 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915359 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915362 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915364 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915366 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915369 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915372 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915375 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915377 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915381 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915383 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915386 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915389 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915391 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 14:16:56.920156 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915394 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915396 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915398 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915401 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915405 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915408 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915411 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915414 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915417 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915419 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915422 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915425 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915428 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915430 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915433 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915436 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915439 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915441 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915444 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 14:16:56.920729 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915447 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915449 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915452 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915454 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915457 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915459 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915462 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915465 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915468 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915471 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915474 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915476 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915479 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915481 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915484 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915486 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915489 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915491 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915494 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915503 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 14:16:56.921192 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915506 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915509 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915511 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915514 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915517 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915519 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915521 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915524 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915528 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915531 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915534 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915536 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915539 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915541 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915544 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915546 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915548 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915551 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915554 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915557 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 14:16:56.921704 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915559 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915561 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915565 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915568 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915570 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915573 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:56.915575 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.915580 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.915705 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.917834 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.919189 2555 server.go:1019] "Starting client certificate rotation" Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.919289 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.919322 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 14:16:56.922221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.921583 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 14:16:56.927234 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.927211 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 14:16:56.934988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.934967 2555 log.go:25] "Validated CRI v1 runtime API" Jul 27 14:16:56.937177 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.937161 2555 log.go:25] "Validated CRI v1 image API" Jul 27 14:16:56.937970 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.937950 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 27 14:16:56.940874 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.940846 2555 fs.go:135] Filesystem UUIDs: map[10f99ebf-ada1-4f3d-9dd5-0607d4a168c5:/dev/nvme0n1p4 28c1e4cd-d686-4dab-b5d0-4fbbd398a3af:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 27 14:16:56.940930 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.940873 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 27 14:16:56.945970 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.945867 2555 manager.go:216] Machine: {Timestamp:2026-07-27 14:16:56.944589774 +0000 UTC m=+0.311851812 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3130280 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20c0142aa990e18e12f1b5417f6d95 SystemUUID:ec20c014-2aa9-90e1-8e12-f1b5417f6d95 BootID:3bd3590a-6f4c-415f-b15e-b4b8a9c7ee1b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5c:ea:f5:fd:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5c:ea:f5:fd:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:82:e3:49:73:a1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 27 14:16:56.945970 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.945967 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 27 14:16:56.946062 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.946045 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 27 14:16:56.949068 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.949046 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 27 14:16:56.949205 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.949072 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-223.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 27 14:16:56.950351 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.950339 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 27 14:16:56.950385 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.950358 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 27 14:16:56.950385 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.950366 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 14:16:56.950978 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.950967 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 14:16:56.952389 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.952379 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 27 14:16:56.956399 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.956387 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 27 14:16:56.956516 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.956507 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 27 14:16:56.956545 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.956535 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 27 14:16:56.956572 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.956546 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 27 14:16:56.956572 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.956563 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 27 14:16:56.956572 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.956572 2555 kubelet.go:398] "Adding apiserver pod source" Jul 27 14:16:56.956733 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.956583 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 27 14:16:56.957004 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.956992 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 14:16:56.957044 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.957013 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 14:16:56.957198 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.957175 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 14:16:56.958971 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.958956 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 27 14:16:56.960392 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.960378 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 27 14:16:56.960440 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.960409 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 27 14:16:56.961171 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961160 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 27 14:16:56.961204 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961176 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 27 14:16:56.961204 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961183 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 27 14:16:56.961204 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961190 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 27 14:16:56.961204 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961203 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 27 14:16:56.961317 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961212 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 27 14:16:56.961317 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961221 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 27 14:16:56.961317 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961226 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 27 14:16:56.961317 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961232 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 27 14:16:56.961317 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961238 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 27 14:16:56.961317 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961247 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 27 14:16:56.961317 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.961253 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 27 14:16:56.962145 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.962136 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 27 14:16:56.962182 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.962146 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 27 14:16:56.965785 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.965772 2555 server.go:1268] "Started kubelet" Jul 27 14:16:56.965882 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.965852 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 27 14:16:56.965963 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.965855 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 27 14:16:56.968814 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.968789 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 27 14:16:56.969684 ip-10-0-133-223 systemd[1]: Started Kubernetes Kubelet. Jul 27 14:16:56.970585 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.970565 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-223.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 27 14:16:56.970995 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:56.970919 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 27 14:16:56.971084 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.970930 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 27 14:16:56.971135 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:56.970924 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 27 14:16:56.971653 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.971633 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 27 14:16:56.971776 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:56.971753 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 27 14:16:56.974574 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:56.973802 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-223.ec2.internal.18c62b36ded58331 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-223.ec2.internal,UID:ip-10-0-133-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-223.ec2.internal,},FirstTimestamp:2026-07-27 14:16:56.965743409 +0000 UTC m=+0.333005449,LastTimestamp:2026-07-27 14:16:56.965743409 +0000 UTC m=+0.333005449,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-223.ec2.internal,}" Jul 27 14:16:56.979470 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.979453 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 27 14:16:56.979556 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.979472 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 27 14:16:56.979621 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:56.979573 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:56.980090 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980071 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 27 14:16:56.980173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980093 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 27 14:16:56.980173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980073 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 27 14:16:56.980265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980235 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 27 14:16:56.980265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980245 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 27 14:16:56.980351 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980270 2555 factory.go:55] Registering systemd factory Jul 27 14:16:56.980351 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980287 2555 factory.go:223] Registration of the systemd container factory successfully Jul 27 14:16:56.980539 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980520 2555 factory.go:153] Registering CRI-O factory Jul 27 14:16:56.980539 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980539 2555 factory.go:223] Registration of the crio container factory successfully Jul 27 14:16:56.980689 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980594 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 27 14:16:56.980689 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980612 2555 factory.go:103] Registering Raw factory Jul 27 14:16:56.980689 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.980626 2555 manager.go:1196] Started watching for new ooms in manager Jul 27 14:16:56.981032 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.981016 2555 manager.go:349] Starting recovery of all containers Jul 27 14:16:56.991223 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:56.991192 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 27 14:16:56.991352 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:56.991328 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 27 14:16:56.992164 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.992149 2555 manager.go:354] Recovery completed Jul 27 14:16:56.996423 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.996409 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 14:16:56.998503 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.998481 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory" Jul 27 14:16:56.998585 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.998516 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 14:16:56.998585 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.998531 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID" Jul 27 14:16:56.999085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.999071 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 27 14:16:56.999085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.999084 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 27 14:16:56.999219 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.999100 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 27 14:16:56.999219 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.999076 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 14:16:56.999898 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.999882 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory" Jul 27 14:16:56.999976 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.999911 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 14:16:56.999976 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:56.999925 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID" Jul 27 14:16:57.000764 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.000680 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-223.ec2.internal.18c62b36e0c95826 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-223.ec2.internal,UID:ip-10-0-133-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-223.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-223.ec2.internal,},FirstTimestamp:2026-07-27 14:16:56.99850039 +0000 UTC m=+0.365762433,LastTimestamp:2026-07-27 14:16:56.99850039 +0000 UTC m=+0.365762433,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-223.ec2.internal,}" Jul 27 14:16:57.014229 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.014024 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-223.ec2.internal.18c62b36e0c9b0fc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-223.ec2.internal,UID:ip-10-0-133-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-223.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-223.ec2.internal,},FirstTimestamp:2026-07-27 14:16:56.998523132 +0000 UTC m=+0.365785174,LastTimestamp:2026-07-27 14:16:56.998523132 +0000 UTC m=+0.365785174,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-223.ec2.internal,}" Jul 27 14:16:57.017282 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.017262 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mczzn" Jul 27 14:16:57.026633 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.026562 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-223.ec2.internal.18c62b36e0c9e972 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-223.ec2.internal,UID:ip-10-0-133-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-223.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-223.ec2.internal,},FirstTimestamp:2026-07-27 14:16:56.998537586 +0000 UTC m=+0.365799629,LastTimestamp:2026-07-27 14:16:56.998537586 +0000 UTC m=+0.365799629,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-223.ec2.internal,}" Jul 27 14:16:57.027103 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.027085 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mczzn" Jul 27 14:16:57.038945 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.038880 2555 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-223.ec2.internal.18c62b36e0c95826\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-223.ec2.internal.18c62b36e0c95826 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-223.ec2.internal,UID:ip-10-0-133-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-223.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-223.ec2.internal,},FirstTimestamp:2026-07-27 14:16:56.99850039 +0000 UTC m=+0.365762433,LastTimestamp:2026-07-27 14:16:56.999897584 +0000 UTC m=+0.367159627,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-223.ec2.internal,}" Jul 27 14:16:57.054688 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.054603 2555 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-223.ec2.internal.18c62b36e0c9b0fc\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-223.ec2.internal.18c62b36e0c9b0fc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-223.ec2.internal,UID:ip-10-0-133-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-223.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-223.ec2.internal,},FirstTimestamp:2026-07-27 14:16:56.998523132 +0000 UTC m=+0.365785174,LastTimestamp:2026-07-27 14:16:56.999918774 +0000 UTC m=+0.367180812,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-223.ec2.internal,}" Jul 27 14:16:57.067629 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.067538 2555 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-223.ec2.internal.18c62b36e0c9e972\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-223.ec2.internal.18c62b36e0c9e972 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-223.ec2.internal,UID:ip-10-0-133-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-223.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-223.ec2.internal,},FirstTimestamp:2026-07-27 14:16:56.998537586 +0000 UTC m=+0.365799629,LastTimestamp:2026-07-27 14:16:56.999929054 +0000 UTC m=+0.367191092,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-223.ec2.internal,}" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.080612 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.140424 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.141644 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.141694 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.141722 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.141759 2555 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.143993 2555 policy_none.go:49] "None policy: Start" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.144015 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 27 14:16:57.145001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.144033 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 27 14:16:57.146245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.146233 2555 policy_none.go:47] "Start" Jul 27 14:16:57.151029 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.150980 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 14:16:57.181554 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.181536 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.183208 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.183195 2555 manager.go:337] "Starting Device Plugin manager" Jul 27 14:16:57.183270 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.183226 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 27 14:16:57.183270 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.183236 2555 server.go:87] "Starting device plugin registration server" Jul 27 14:16:57.183535 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.183515 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 27 14:16:57.183619 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.183530 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 27 14:16:57.183714 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.183700 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 27 14:16:57.183799 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.183771 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 27 14:16:57.183799 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.183779 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 27 14:16:57.184330 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.184304 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 27 14:16:57.184400 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.184351 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.196619 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.196589 2555 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-223.ec2.internal\" not found" node="ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.242758 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.242722 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal"] Jul 27 14:16:57.242879 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.242821 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 14:16:57.243816 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.243800 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory" Jul 27 14:16:57.243894 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.243828 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 14:16:57.243894 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.243837 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID" Jul 27 14:16:57.246293 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.246280 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 14:16:57.246417 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.246402 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.246467 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.246432 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 14:16:57.247059 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.247045 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory" Jul 27 14:16:57.247170 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.247071 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 14:16:57.247170 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.247044 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory" Jul 27 14:16:57.247170 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.247114 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 14:16:57.247170 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.247085 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID" Jul 27 14:16:57.247170 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.247131 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID" Jul 27 14:16:57.249439 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.249423 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.249523 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.249452 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 14:16:57.250133 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.250116 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory" Jul 27 14:16:57.250218 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.250147 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 14:16:57.250218 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.250162 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID" Jul 27 14:16:57.270229 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.270209 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-223.ec2.internal\" not found" node="ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.274768 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.274753 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-223.ec2.internal\" not found" node="ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.282189 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.282174 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77040d035aff638adc39fd418cfe0d87-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"77040d035aff638adc39fd418cfe0d87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.282251 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.282198 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77040d035aff638adc39fd418cfe0d87-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"77040d035aff638adc39fd418cfe0d87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.282251 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.282214 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91823a60abd6f5026e7827f004ecd18b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-223.ec2.internal\" (UID: \"91823a60abd6f5026e7827f004ecd18b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.283595 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.283581 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 14:16:57.284743 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.284727 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientMemory" Jul 27 14:16:57.284812 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.284752 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 14:16:57.284812 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.284762 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeHasSufficientPID" Jul 27 14:16:57.284812 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.284789 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.290956 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.290941 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.291041 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.290964 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-223.ec2.internal\": node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.317597 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.317575 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.382986 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.382952 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77040d035aff638adc39fd418cfe0d87-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"77040d035aff638adc39fd418cfe0d87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.383165 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.382984 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77040d035aff638adc39fd418cfe0d87-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"77040d035aff638adc39fd418cfe0d87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.383165 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.383016 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91823a60abd6f5026e7827f004ecd18b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-223.ec2.internal\" (UID: \"91823a60abd6f5026e7827f004ecd18b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.383165 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.383065 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91823a60abd6f5026e7827f004ecd18b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-223.ec2.internal\" (UID: \"91823a60abd6f5026e7827f004ecd18b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.383165 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.383072 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77040d035aff638adc39fd418cfe0d87-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"77040d035aff638adc39fd418cfe0d87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.383165 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.383072 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77040d035aff638adc39fd418cfe0d87-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal\" (UID: \"77040d035aff638adc39fd418cfe0d87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.418145 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.418068 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.518875 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.518846 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.573074 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.573041 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.578740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.578679 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.631196 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.631161 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.731712 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.731635 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.832140 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.832112 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-223.ec2.internal\" not found" Jul 27 14:16:57.876052 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.876026 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 14:16:57.880509 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.880491 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.892885 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.892865 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 14:16:57.894449 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.894437 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" Jul 27 14:16:57.902942 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.902923 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 14:16:57.919052 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.919023 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 27 14:16:57.919151 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.919135 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 14:16:57.919201 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.919169 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 14:16:57.956800 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.956771 2555 apiserver.go:52] "Watching apiserver" Jul 27 14:16:57.977511 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.977486 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 27 14:16:57.978689 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.978667 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-tl44d","openshift-network-operator/iptables-alerter-nqrnp","kube-system/global-pull-secret-syncer-np467","kube-system/konnectivity-agent-k696n","kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49","openshift-cluster-node-tuning-operator/tuned-fvnrd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-xzr9g","openshift-dns/node-resolver-jbjwv","openshift-image-registry/node-ca-gpfpw","openshift-multus/multus-additional-cni-plugins-87smn","openshift-multus/multus-crs6s","openshift-multus/network-metrics-daemon-lxpdl"] Jul 27 14:16:57.979809 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.979792 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 27 14:16:57.981411 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.981392 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.983830 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.983742 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:57.984128 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.984109 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 27 14:16:57.984211 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.984115 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 27 14:16:57.984211 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.984196 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jmtkd\"" Jul 27 14:16:57.986026 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986008 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-modprobe-d\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986033 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-kubernetes\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986049 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-sys\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986077 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysconfig\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986097 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:57.986174 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986100 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysctl-d\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986217 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-var-lib-kubelet\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986228 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986238 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-host\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986263 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvf9h\" (UniqueName: \"kubernetes.io/projected/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-kube-api-access-nvf9h\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986289 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysctl-conf\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986331 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-run\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986357 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-tmp\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986816 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986377 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-systemd\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986816 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986415 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-lib-modules\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986816 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986432 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-tuned\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:57.986816 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986511 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 27 14:16:57.986816 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986740 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 27 14:16:57.986983 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.986897 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zl269\"" Jul 27 14:16:57.988429 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.988415 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k696n" Jul 27 14:16:57.990592 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.990578 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 27 14:16:57.990756 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.990702 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:57.991031 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.991014 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 27 14:16:57.991129 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.991063 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k6nwf\"" Jul 27 14:16:57.991437 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.991418 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 14:16:57.993126 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.993108 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:57.993474 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.993460 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 27 14:16:57.993646 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.993631 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-n267q\"" Jul 27 14:16:57.993911 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.993894 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 27 14:16:57.995453 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.995436 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:57.995893 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.995876 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 27 14:16:57.995990 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.995956 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mrzk6\"" Jul 27 14:16:57.996050 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.996034 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 27 14:16:57.996481 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.996463 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 27 14:16:57.997816 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.997798 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 27 14:16:57.997897 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.997854 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-crs6s" Jul 27 14:16:57.998268 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.998240 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lcnk7\"" Jul 27 14:16:57.998335 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.998293 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 27 14:16:57.998397 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:57.998384 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 27 14:16:58.000447 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.000424 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.000852 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.000749 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wg8wk\"" Jul 27 14:16:58.001219 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.001203 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 27 14:16:58.001277 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.001262 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 27 14:16:58.001326 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.001286 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 27 14:16:58.001463 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.001449 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 27 14:16:58.002960 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.002941 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.003159 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.003002 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:16:58.003159 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.003022 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 27 14:16:58.003272 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.003228 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 27 14:16:58.003342 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.003323 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 27 14:16:58.006394 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.006199 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 27 14:16:58.006394 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.006228 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9mpxl\"" Jul 27 14:16:58.006394 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.006247 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 27 14:16:58.006394 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.006267 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 27 14:16:58.009376 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.009357 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.010008 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.009989 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:16:58.010084 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.010054 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:16:58.013082 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.013064 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 27 14:16:58.013177 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.013099 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 27 14:16:58.013177 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.013116 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2n2ds\"" Jul 27 14:16:58.022381 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.022360 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nd24j" Jul 27 14:16:58.032180 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.032159 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nd24j" Jul 27 14:16:58.081855 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.081836 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 27 14:16:58.087437 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087412 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7272bcef-4347-40ac-9ce4-9de68958d689-multus-daemon-config\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.087532 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087448 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-lib-modules\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.087532 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087472 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-socket-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.087532 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087504 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmch\" (UniqueName: \"kubernetes.io/projected/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-kube-api-access-5pmch\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.087532 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087525 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d36d1-a3d2-477c-b22a-d7963daf4fb1-konnectivity-ca\") pod \"konnectivity-agent-k696n\" (UID: \"4b2d36d1-a3d2-477c-b22a-d7963daf4fb1\") " pod="kube-system/konnectivity-agent-k696n" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087545 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdkq8\" (UniqueName: \"kubernetes.io/projected/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-kube-api-access-hdkq8\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087568 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-var-lib-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087602 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-cni-bin\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087603 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-lib-modules\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087645 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-cnibin\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087690 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-netns\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087718 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087769 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-var-lib-kubelet\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.087831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087801 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-host\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087839 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-etc-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087864 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-hostroot\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087875 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-host\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087889 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087876 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-var-lib-kubelet\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087924 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-system-cni-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087946 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9b27844f-7fba-4449-b514-1f2d9424f97e-dbus\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087967 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-env-overrides\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.087991 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088011 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-log-socket\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088035 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-cni-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088050 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-conf-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088073 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088099 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088122 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgkpl\" (UniqueName: \"kubernetes.io/projected/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-kube-api-access-cgkpl\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:16:58.088163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysctl-d\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088161 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxlmb\" (UniqueName: \"kubernetes.io/projected/def06289-55b1-475f-a338-42014932d174-kube-api-access-zxlmb\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088181 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-sys-fs\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088199 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-systemd-units\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088214 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-node-log\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088241 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b2d36d1-a3d2-477c-b22a-d7963daf4fb1-agent-certs\") pod \"konnectivity-agent-k696n\" (UID: \"4b2d36d1-a3d2-477c-b22a-d7963daf4fb1\") " pod="kube-system/konnectivity-agent-k696n" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088255 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysctl-d\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088263 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-host-slash\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088308 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-cni-bin\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088334 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-run\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088359 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-tmp-dir\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088383 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088396 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-run\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088409 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-ovnkube-script-lib\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088435 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7272bcef-4347-40ac-9ce4-9de68958d689-cni-binary-copy\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088459 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-multus-certs\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088486 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-tuned\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.088988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088510 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-system-cni-dir\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088536 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-os-release\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088561 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nmlm\" (UniqueName: \"kubernetes.io/projected/7272bcef-4347-40ac-9ce4-9de68958d689-kube-api-access-8nmlm\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088604 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-modprobe-d\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088636 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-sys\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088676 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-slash\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088737 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-modprobe-d\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088725 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-sys\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-cni-netd\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088834 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088852 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-cni-binary-copy\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088891 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-socket-dir-parent\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088921 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvf9h\" (UniqueName: \"kubernetes.io/projected/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-kube-api-access-nvf9h\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088945 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4wxm\" (UniqueName: \"kubernetes.io/projected/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-kube-api-access-s4wxm\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.088971 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-run-netns\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089010 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-run-ovn-kubernetes\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089037 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38423966-1ba9-464f-8227-256adcb6b870-ovn-node-metrics-cert\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.089661 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089061 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-cni-multus\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089089 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysctl-conf\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089113 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-tmp\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089137 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/def06289-55b1-475f-a338-42014932d174-serviceca\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089191 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysctl-conf\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089236 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-kubelet\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089266 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-systemd\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089292 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-ovn\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089327 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-ovnkube-config\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089353 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-tuning-conf-dir\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089383 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-registration-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089411 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-systemd\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089433 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/def06289-55b1-475f-a338-42014932d174-host\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089452 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-hosts-file\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089467 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089501 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-iptables-alerter-script\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089522 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-cnibin\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.090265 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089518 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-systemd\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089539 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-kubernetes\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089590 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-kubernetes\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089602 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-device-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089630 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rdlb\" (UniqueName: \"kubernetes.io/projected/38423966-1ba9-464f-8227-256adcb6b870-kube-api-access-7rdlb\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc79x\" (UniqueName: \"kubernetes.io/projected/acf16aa1-2fc6-4275-9845-a26701408128-kube-api-access-fc79x\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089732 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9b27844f-7fba-4449-b514-1f2d9424f97e-kubelet-config\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089774 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089803 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089826 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysconfig\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089873 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-os-release\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089880 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-sysconfig\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089907 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-k8s-cni-cncf-io\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089930 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-kubelet\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.090937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.089946 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-etc-kubernetes\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.091844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.091825 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-etc-tuned\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.091927 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.091887 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-tmp\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.097887 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.097865 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvf9h\" (UniqueName: \"kubernetes.io/projected/eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3-kube-api-access-nvf9h\") pod \"tuned-fvnrd\" (UID: \"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.102451 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.102430 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 14:16:58.147665 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.147595 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 14:11:57 +0000 UTC" deadline="2028-03-31 09:38:25.405252676 +0000 UTC" Jul 27 14:16:58.147842 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.147680 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14707h21m27.257594461s" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190256 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-kubelet\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190300 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-systemd\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190324 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-ovn\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190348 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-ovnkube-config\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-tuning-conf-dir\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190415 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-registration-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/def06289-55b1-475f-a338-42014932d174-host\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190473 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-hosts-file\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190498 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-iptables-alerter-script\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190545 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-cnibin\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.190571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190569 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-device-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190590 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rdlb\" (UniqueName: \"kubernetes.io/projected/38423966-1ba9-464f-8227-256adcb6b870-kube-api-access-7rdlb\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190615 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc79x\" (UniqueName: \"kubernetes.io/projected/acf16aa1-2fc6-4275-9845-a26701408128-kube-api-access-fc79x\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190637 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9b27844f-7fba-4449-b514-1f2d9424f97e-kubelet-config\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190646 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-kubelet\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190660 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190720 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190745 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-os-release\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190773 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-k8s-cni-cncf-io\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.190783 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190791 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-kubelet\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190815 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-etc-kubernetes\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190836 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7272bcef-4347-40ac-9ce4-9de68958d689-multus-daemon-config\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.190859 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret podName:9b27844f-7fba-4449-b514-1f2d9424f97e nodeName:}" failed. No retries permitted until 2026-07-27 14:16:58.690834401 +0000 UTC m=+2.058096451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret") pod "global-pull-secret-syncer-np467" (UID: "9b27844f-7fba-4449-b514-1f2d9424f97e") : object "kube-system"/"original-pull-secret" not registered Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190855 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/def06289-55b1-475f-a338-42014932d174-host\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190891 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-socket-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190914 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-registration-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.191323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190935 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-hosts-file\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190922 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmch\" (UniqueName: \"kubernetes.io/projected/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-kube-api-access-5pmch\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190979 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d36d1-a3d2-477c-b22a-d7963daf4fb1-konnectivity-ca\") pod \"konnectivity-agent-k696n\" (UID: \"4b2d36d1-a3d2-477c-b22a-d7963daf4fb1\") " pod="kube-system/konnectivity-agent-k696n" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.190984 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-device-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191008 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdkq8\" (UniqueName: \"kubernetes.io/projected/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-kube-api-access-hdkq8\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-var-lib-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191055 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-cni-bin\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191081 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-cnibin\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191103 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-netns\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191128 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191184 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-cnibin\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191222 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-etc-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191252 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-hostroot\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191279 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191305 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-system-cni-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191326 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9b27844f-7fba-4449-b514-1f2d9424f97e-dbus\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191349 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-env-overrides\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.192245 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.191375 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-log-socket\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-cni-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.191450 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs podName:a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de nodeName:}" failed. No retries permitted until 2026-07-27 14:16:58.691429906 +0000 UTC m=+2.058691958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs") pod "network-metrics-daemon-lxpdl" (UID: "a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191450 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7272bcef-4347-40ac-9ce4-9de68958d689-multus-daemon-config\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191478 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-cni-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191480 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9b27844f-7fba-4449-b514-1f2d9424f97e-kubelet-config\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191490 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-conf-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191517 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191537 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-os-release\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191544 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191548 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-hostroot\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191580 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-k8s-cni-cncf-io\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191597 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9b27844f-7fba-4449-b514-1f2d9424f97e-dbus\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191606 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-systemd\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191628 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-system-cni-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191639 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-ovn\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.193185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191671 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191685 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-kubelet\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191629 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgkpl\" (UniqueName: \"kubernetes.io/projected/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-kube-api-access-cgkpl\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191724 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191750 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxlmb\" (UniqueName: \"kubernetes.io/projected/def06289-55b1-475f-a338-42014932d174-kube-api-access-zxlmb\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191759 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-socket-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191781 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-sys-fs\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191807 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-systemd-units\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191830 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-node-log\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191903 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b2d36d1-a3d2-477c-b22a-d7963daf4fb1-agent-certs\") pod \"konnectivity-agent-k696n\" (UID: \"4b2d36d1-a3d2-477c-b22a-d7963daf4fb1\") " pod="kube-system/konnectivity-agent-k696n" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191926 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-host-slash\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191948 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-cni-bin\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191974 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-tmp-dir\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191975 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-etc-kubernetes\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191991 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-tuning-conf-dir\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192004 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192044 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-ovnkube-script-lib\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.193844 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192051 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-run-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192054 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192076 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7272bcef-4347-40ac-9ce4-9de68958d689-cni-binary-copy\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191675 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-log-socket\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192116 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-sys-fs\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192173 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192228 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-conf-dir\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192287 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-ovnkube-config\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.191518 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-etc-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192345 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-host-slash\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192395 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-cni-bin\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192474 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-env-overrides\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192486 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-systemd-units\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192549 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-var-lib-openvswitch\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192552 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-node-log\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192651 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38423966-1ba9-464f-8227-256adcb6b870-ovnkube-script-lib\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192683 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-tmp-dir\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192654 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.194412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192722 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-iptables-alerter-script\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192754 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-netns\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192738 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-cni-bin\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192777 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-multus-certs\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192814 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-system-cni-dir\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192831 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-run-multus-certs\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-os-release\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192876 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nmlm\" (UniqueName: \"kubernetes.io/projected/7272bcef-4347-40ac-9ce4-9de68958d689-kube-api-access-8nmlm\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192893 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-os-release\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192908 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-slash\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192934 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-cni-netd\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192939 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-system-cni-dir\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192783 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acf16aa1-2fc6-4275-9845-a26701408128-cnibin\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192979 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-cni-netd\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.192985 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-cni-binary-copy\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193013 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-socket-dir-parent\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193024 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7272bcef-4347-40ac-9ce4-9de68958d689-cni-binary-copy\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193043 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4wxm\" (UniqueName: \"kubernetes.io/projected/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-kube-api-access-s4wxm\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193060 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d36d1-a3d2-477c-b22a-d7963daf4fb1-konnectivity-ca\") pod \"konnectivity-agent-k696n\" (UID: \"4b2d36d1-a3d2-477c-b22a-d7963daf4fb1\") " pod="kube-system/konnectivity-agent-k696n" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193014 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-slash\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193099 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-multus-socket-dir-parent\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193129 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-run-netns\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193148 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-run-ovn-kubernetes\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193172 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38423966-1ba9-464f-8227-256adcb6b870-ovn-node-metrics-cert\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193200 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-cni-multus\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193208 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-run-netns\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193227 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38423966-1ba9-464f-8227-256adcb6b870-host-run-ovn-kubernetes\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193229 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/def06289-55b1-475f-a338-42014932d174-serviceca\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193268 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7272bcef-4347-40ac-9ce4-9de68958d689-host-var-lib-cni-multus\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193421 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acf16aa1-2fc6-4275-9845-a26701408128-cni-binary-copy\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.193600 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/def06289-55b1-475f-a338-42014932d174-serviceca\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.194878 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b2d36d1-a3d2-477c-b22a-d7963daf4fb1-agent-certs\") pod \"konnectivity-agent-k696n\" (UID: \"4b2d36d1-a3d2-477c-b22a-d7963daf4fb1\") " pod="kube-system/konnectivity-agent-k696n" Jul 27 14:16:58.195685 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.195645 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38423966-1ba9-464f-8227-256adcb6b870-ovn-node-metrics-cert\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.203833 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.203806 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 14:16:58.203833 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.203834 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 14:16:58.204040 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.203848 2555 projected.go:196] Error preparing data for projected volume kube-api-access-2mj97 for pod openshift-network-diagnostics/network-check-target-tl44d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:16:58.204040 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.203930 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97 podName:0b1c2c00-b265-42b8-b25c-28a4acd4f3c7 nodeName:}" failed. No retries permitted until 2026-07-27 14:16:58.703914182 +0000 UTC m=+2.071176214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2mj97" (UniqueName: "kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97") pod "network-check-target-tl44d" (UID: "0b1c2c00-b265-42b8-b25c-28a4acd4f3c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:16:58.204847 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.204803 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc79x\" (UniqueName: \"kubernetes.io/projected/acf16aa1-2fc6-4275-9845-a26701408128-kube-api-access-fc79x\") pod \"multus-additional-cni-plugins-87smn\" (UID: \"acf16aa1-2fc6-4275-9845-a26701408128\") " pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.205173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.205155 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rdlb\" (UniqueName: \"kubernetes.io/projected/38423966-1ba9-464f-8227-256adcb6b870-kube-api-access-7rdlb\") pod \"ovnkube-node-xzr9g\" (UID: \"38423966-1ba9-464f-8227-256adcb6b870\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.206158 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.206132 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmch\" (UniqueName: \"kubernetes.io/projected/c93e7038-cc5e-47f1-a26e-3d50a7eb963e-kube-api-access-5pmch\") pod \"aws-ebs-csi-driver-node-d6l49\" (UID: \"c93e7038-cc5e-47f1-a26e-3d50a7eb963e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.206479 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.206445 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdkq8\" (UniqueName: \"kubernetes.io/projected/d78abc26-4e7e-4509-befb-3e19f7dc9b3e-kube-api-access-hdkq8\") pod \"iptables-alerter-nqrnp\" (UID: \"d78abc26-4e7e-4509-befb-3e19f7dc9b3e\") " pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.206738 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.206674 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxlmb\" (UniqueName: \"kubernetes.io/projected/def06289-55b1-475f-a338-42014932d174-kube-api-access-zxlmb\") pod \"node-ca-gpfpw\" (UID: \"def06289-55b1-475f-a338-42014932d174\") " pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.207160 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.207135 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nmlm\" (UniqueName: \"kubernetes.io/projected/7272bcef-4347-40ac-9ce4-9de68958d689-kube-api-access-8nmlm\") pod \"multus-crs6s\" (UID: \"7272bcef-4347-40ac-9ce4-9de68958d689\") " pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.207600 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.207583 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4wxm\" (UniqueName: \"kubernetes.io/projected/4bdecfbd-3e2f-47f0-a85b-2efffb7aef68-kube-api-access-s4wxm\") pod \"node-resolver-jbjwv\" (UID: \"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68\") " pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.207865 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.207847 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgkpl\" (UniqueName: \"kubernetes.io/projected/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-kube-api-access-cgkpl\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:16:58.214267 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.214250 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 14:16:58.254518 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.254479 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91823a60abd6f5026e7827f004ecd18b.slice/crio-1b0ccf80b7e4781d9bde657ac25dc59032af85993e03c837c242faaef9d98924 WatchSource:0}: Error finding container 1b0ccf80b7e4781d9bde657ac25dc59032af85993e03c837c242faaef9d98924: Status 404 returned error can't find the container with id 1b0ccf80b7e4781d9bde657ac25dc59032af85993e03c837c242faaef9d98924 Jul 27 14:16:58.254751 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.254730 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77040d035aff638adc39fd418cfe0d87.slice/crio-aeee7a9f3365a86b5c71ce175cdfedc9de5e819376fb792a87cb1bba13448fb9 WatchSource:0}: Error finding container aeee7a9f3365a86b5c71ce175cdfedc9de5e819376fb792a87cb1bba13448fb9: Status 404 returned error can't find the container with id aeee7a9f3365a86b5c71ce175cdfedc9de5e819376fb792a87cb1bba13448fb9 Jul 27 14:16:58.259187 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.259173 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 14:16:58.308622 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.308584 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" Jul 27 14:16:58.319273 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.319246 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8cb916_12e7_40eb_a9ce_2adf3ff30ca3.slice/crio-60495a37267dbc1eff84950a65fef81b3fff8c92562334280467621bdd03f94a WatchSource:0}: Error finding container 60495a37267dbc1eff84950a65fef81b3fff8c92562334280467621bdd03f94a: Status 404 returned error can't find the container with id 60495a37267dbc1eff84950a65fef81b3fff8c92562334280467621bdd03f94a Jul 27 14:16:58.326662 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.326643 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gpfpw" Jul 27 14:16:58.333839 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.333817 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k696n" Jul 27 14:16:58.334727 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.334709 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef06289_55b1_475f_a338_42014932d174.slice/crio-1c9b3797e8873e5ed40f5650431202921b4b0a62c9e8e840bd94888ef69f67f8 WatchSource:0}: Error finding container 1c9b3797e8873e5ed40f5650431202921b4b0a62c9e8e840bd94888ef69f67f8: Status 404 returned error can't find the container with id 1c9b3797e8873e5ed40f5650431202921b4b0a62c9e8e840bd94888ef69f67f8 Jul 27 14:16:58.343828 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.343811 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jbjwv" Jul 27 14:16:58.344066 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.344048 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2d36d1_a3d2_477c_b22a_d7963daf4fb1.slice/crio-f5322c846d2104ec5173c1837626475f33280d21ae9bb18a7740fa197ab92d30 WatchSource:0}: Error finding container f5322c846d2104ec5173c1837626475f33280d21ae9bb18a7740fa197ab92d30: Status 404 returned error can't find the container with id f5322c846d2104ec5173c1837626475f33280d21ae9bb18a7740fa197ab92d30 Jul 27 14:16:58.350860 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.350840 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" Jul 27 14:16:58.352581 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.352559 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdecfbd_3e2f_47f0_a85b_2efffb7aef68.slice/crio-61775c13bc288048893088fa576abf2d02af255aed21c3b27df04a7fff8e144c WatchSource:0}: Error finding container 61775c13bc288048893088fa576abf2d02af255aed21c3b27df04a7fff8e144c: Status 404 returned error can't find the container with id 61775c13bc288048893088fa576abf2d02af255aed21c3b27df04a7fff8e144c Jul 27 14:16:58.360103 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.360085 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93e7038_cc5e_47f1_a26e_3d50a7eb963e.slice/crio-eaccfc9f7b1a19facb36c895a53f289866aa0b7a0abb2f428bec54413199e759 WatchSource:0}: Error finding container eaccfc9f7b1a19facb36c895a53f289866aa0b7a0abb2f428bec54413199e759: Status 404 returned error can't find the container with id eaccfc9f7b1a19facb36c895a53f289866aa0b7a0abb2f428bec54413199e759 Jul 27 14:16:58.366618 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.366601 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nqrnp" Jul 27 14:16:58.374650 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.374611 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78abc26_4e7e_4509_befb_3e19f7dc9b3e.slice/crio-50aadec9d16eea5d7081452e4a2425eb4f1255945bd87f68464ea6cdf4370928 WatchSource:0}: Error finding container 50aadec9d16eea5d7081452e4a2425eb4f1255945bd87f68464ea6cdf4370928: Status 404 returned error can't find the container with id 50aadec9d16eea5d7081452e4a2425eb4f1255945bd87f68464ea6cdf4370928 Jul 27 14:16:58.381811 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.381775 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-crs6s" Jul 27 14:16:58.389703 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.389683 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7272bcef_4347_40ac_9ce4_9de68958d689.slice/crio-bb9445841cb066370c1e9587d98433ad266b3c6d6bd168634a3b98575abf3c69 WatchSource:0}: Error finding container bb9445841cb066370c1e9587d98433ad266b3c6d6bd168634a3b98575abf3c69: Status 404 returned error can't find the container with id bb9445841cb066370c1e9587d98433ad266b3c6d6bd168634a3b98575abf3c69 Jul 27 14:16:58.393103 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.393084 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:16:58.401019 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.400999 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38423966_1ba9_464f_8227_256adcb6b870.slice/crio-67ac0d969c495810e6f6a7f205565a91de6a49d96c08ac59fb0e2a2670e53c04 WatchSource:0}: Error finding container 67ac0d969c495810e6f6a7f205565a91de6a49d96c08ac59fb0e2a2670e53c04: Status 404 returned error can't find the container with id 67ac0d969c495810e6f6a7f205565a91de6a49d96c08ac59fb0e2a2670e53c04 Jul 27 14:16:58.405104 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.405087 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-87smn" Jul 27 14:16:58.414120 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:16:58.414098 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf16aa1_2fc6_4275_9845_a26701408128.slice/crio-77a5f01a260e425123cabdc4607e689d0e27c221397bef09b3e495e56d124431 WatchSource:0}: Error finding container 77a5f01a260e425123cabdc4607e689d0e27c221397bef09b3e495e56d124431: Status 404 returned error can't find the container with id 77a5f01a260e425123cabdc4607e689d0e27c221397bef09b3e495e56d124431 Jul 27 14:16:58.415959 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.415938 2555 kuberuntime_manager.go:1459] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-87smn_openshift-multus(acf16aa1-2fc6-4275-9845-a26701408128): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 27 14:16:58.416043 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.415977 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-87smn" podUID="acf16aa1-2fc6-4275-9845-a26701408128" Jul 27 14:16:58.697995 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.697890 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:58.697995 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.697938 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:16:58.698212 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.698052 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:16:58.698212 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.698066 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 14:16:58.698212 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.698108 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs podName:a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de nodeName:}" failed. No retries permitted until 2026-07-27 14:16:59.698092901 +0000 UTC m=+3.065354929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs") pod "network-metrics-daemon-lxpdl" (UID: "a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:16:58.698212 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.698146 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret podName:9b27844f-7fba-4449-b514-1f2d9424f97e nodeName:}" failed. No retries permitted until 2026-07-27 14:16:59.698128459 +0000 UTC m=+3.065390510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret") pod "global-pull-secret-syncer-np467" (UID: "9b27844f-7fba-4449-b514-1f2d9424f97e") : object "kube-system"/"original-pull-secret" not registered Jul 27 14:16:58.798469 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:58.798427 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:16:58.798693 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.798652 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 14:16:58.798693 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.798677 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 14:16:58.798693 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.798691 2555 projected.go:196] Error preparing data for projected volume kube-api-access-2mj97 for pod openshift-network-diagnostics/network-check-target-tl44d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:16:58.798953 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:58.798784 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97 podName:0b1c2c00-b265-42b8-b25c-28a4acd4f3c7 nodeName:}" failed. No retries permitted until 2026-07-27 14:16:59.798764016 +0000 UTC m=+3.166026047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mj97" (UniqueName: "kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97") pod "network-check-target-tl44d" (UID: "0b1c2c00-b265-42b8-b25c-28a4acd4f3c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:16:59.038510 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.038393 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 14:11:58 +0000 UTC" deadline="2028-04-10 00:18:35.695405042 +0000 UTC" Jul 27 14:16:59.038510 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.038459 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14938h1m36.656950151s" Jul 27 14:16:59.050054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.049819 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 14:16:59.142832 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.142801 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:16:59.143014 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.142935 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:16:59.177403 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.177341 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerStarted","Data":"77a5f01a260e425123cabdc4607e689d0e27c221397bef09b3e495e56d124431"} Jul 27 14:16:59.184598 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.184564 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-87smn" podUID="acf16aa1-2fc6-4275-9845-a26701408128" Jul 27 14:16:59.207051 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.207019 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nqrnp" event={"ID":"d78abc26-4e7e-4509-befb-3e19f7dc9b3e","Type":"ContainerStarted","Data":"50aadec9d16eea5d7081452e4a2425eb4f1255945bd87f68464ea6cdf4370928"} Jul 27 14:16:59.221630 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.221596 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gpfpw" event={"ID":"def06289-55b1-475f-a338-42014932d174","Type":"ContainerStarted","Data":"1c9b3797e8873e5ed40f5650431202921b4b0a62c9e8e840bd94888ef69f67f8"} Jul 27 14:16:59.250893 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.250860 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" event={"ID":"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3","Type":"ContainerStarted","Data":"60495a37267dbc1eff84950a65fef81b3fff8c92562334280467621bdd03f94a"} Jul 27 14:16:59.273179 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.273146 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" event={"ID":"77040d035aff638adc39fd418cfe0d87","Type":"ContainerStarted","Data":"aeee7a9f3365a86b5c71ce175cdfedc9de5e819376fb792a87cb1bba13448fb9"} Jul 27 14:16:59.284919 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.284848 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" event={"ID":"91823a60abd6f5026e7827f004ecd18b","Type":"ContainerStarted","Data":"1b0ccf80b7e4781d9bde657ac25dc59032af85993e03c837c242faaef9d98924"} Jul 27 14:16:59.302894 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.302728 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"67ac0d969c495810e6f6a7f205565a91de6a49d96c08ac59fb0e2a2670e53c04"} Jul 27 14:16:59.318995 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.318931 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-crs6s" event={"ID":"7272bcef-4347-40ac-9ce4-9de68958d689","Type":"ContainerStarted","Data":"bb9445841cb066370c1e9587d98433ad266b3c6d6bd168634a3b98575abf3c69"} Jul 27 14:16:59.333727 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.333642 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" event={"ID":"c93e7038-cc5e-47f1-a26e-3d50a7eb963e","Type":"ContainerStarted","Data":"eaccfc9f7b1a19facb36c895a53f289866aa0b7a0abb2f428bec54413199e759"} Jul 27 14:16:59.343631 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.343602 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jbjwv" event={"ID":"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68","Type":"ContainerStarted","Data":"61775c13bc288048893088fa576abf2d02af255aed21c3b27df04a7fff8e144c"} Jul 27 14:16:59.361104 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.361017 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k696n" event={"ID":"4b2d36d1-a3d2-477c-b22a-d7963daf4fb1","Type":"ContainerStarted","Data":"f5322c846d2104ec5173c1837626475f33280d21ae9bb18a7740fa197ab92d30"} Jul 27 14:16:59.707598 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.707492 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:16:59.707598 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.707544 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:16:59.707845 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.707734 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:16:59.707845 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.707802 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs podName:a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de nodeName:}" failed. No retries permitted until 2026-07-27 14:17:01.707782656 +0000 UTC m=+5.075044718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs") pod "network-metrics-daemon-lxpdl" (UID: "a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:16:59.708337 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.708314 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 14:16:59.708506 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.708373 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret podName:9b27844f-7fba-4449-b514-1f2d9424f97e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:01.708358425 +0000 UTC m=+5.075620468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret") pod "global-pull-secret-syncer-np467" (UID: "9b27844f-7fba-4449-b514-1f2d9424f97e") : object "kube-system"/"original-pull-secret" not registered Jul 27 14:16:59.808922 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.808252 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:16:59.808922 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.808413 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 14:16:59.808922 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.808438 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 14:16:59.808922 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.808451 2555 projected.go:196] Error preparing data for projected volume kube-api-access-2mj97 for pod openshift-network-diagnostics/network-check-target-tl44d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:16:59.808922 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:16:59.808513 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97 podName:0b1c2c00-b265-42b8-b25c-28a4acd4f3c7 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:01.808493832 +0000 UTC m=+5.175755862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mj97" (UniqueName: "kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97") pod "network-check-target-tl44d" (UID: "0b1c2c00-b265-42b8-b25c-28a4acd4f3c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:16:59.936176 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:16:59.936139 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 14:17:00.038996 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:00.038839 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 14:11:58 +0000 UTC" deadline="2027-12-26 11:59:56.583845477 +0000 UTC" Jul 27 14:17:00.038996 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:00.038890 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12405h42m56.54496135s" Jul 27 14:17:00.143353 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:00.142510 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:00.143353 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:00.142738 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:00.143353 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:00.143196 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:00.143353 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:00.143298 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:01.143860 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:01.143827 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:01.144357 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.143977 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:01.724286 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:01.724123 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:01.724481 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:01.724313 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:01.735455 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.734910 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 14:17:01.735455 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.735012 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret podName:9b27844f-7fba-4449-b514-1f2d9424f97e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:05.734987653 +0000 UTC m=+9.102249681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret") pod "global-pull-secret-syncer-np467" (UID: "9b27844f-7fba-4449-b514-1f2d9424f97e") : object "kube-system"/"original-pull-secret" not registered Jul 27 14:17:01.737913 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.735800 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:17:01.737913 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.735875 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs podName:a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de nodeName:}" failed. No retries permitted until 2026-07-27 14:17:05.735855416 +0000 UTC m=+9.103117445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs") pod "network-metrics-daemon-lxpdl" (UID: "a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:17:01.825927 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:01.825180 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:01.825927 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.825371 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 14:17:01.825927 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.825393 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 14:17:01.825927 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.825406 2555 projected.go:196] Error preparing data for projected volume kube-api-access-2mj97 for pod openshift-network-diagnostics/network-check-target-tl44d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:17:01.825927 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:01.825484 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97 podName:0b1c2c00-b265-42b8-b25c-28a4acd4f3c7 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:05.825463404 +0000 UTC m=+9.192725439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mj97" (UniqueName: "kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97") pod "network-check-target-tl44d" (UID: "0b1c2c00-b265-42b8-b25c-28a4acd4f3c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:17:02.143876 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:02.143100 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:02.143876 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:02.143247 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:02.143876 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:02.143100 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:02.143876 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:02.143768 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:03.142612 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:03.142575 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:03.142819 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:03.142761 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:04.142706 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:04.142676 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:04.143151 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:04.142723 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:04.143151 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:04.142799 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:04.143151 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:04.142940 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:05.142473 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:05.142438 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:05.142685 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.142569 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:05.757762 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:05.757724 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:05.758222 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:05.757868 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:05.758222 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.757902 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 14:17:05.758222 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.757985 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:17:05.758222 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.757990 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret podName:9b27844f-7fba-4449-b514-1f2d9424f97e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:13.757968175 +0000 UTC m=+17.125230201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret") pod "global-pull-secret-syncer-np467" (UID: "9b27844f-7fba-4449-b514-1f2d9424f97e") : object "kube-system"/"original-pull-secret" not registered Jul 27 14:17:05.758222 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.758045 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs podName:a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de nodeName:}" failed. No retries permitted until 2026-07-27 14:17:13.758032907 +0000 UTC m=+17.125294933 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs") pod "network-metrics-daemon-lxpdl" (UID: "a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:17:05.859225 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:05.858505 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:05.859225 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.858692 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 14:17:05.859225 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.858824 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 14:17:05.859225 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.858839 2555 projected.go:196] Error preparing data for projected volume kube-api-access-2mj97 for pod openshift-network-diagnostics/network-check-target-tl44d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:17:05.859225 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:05.858902 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97 podName:0b1c2c00-b265-42b8-b25c-28a4acd4f3c7 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:13.858883145 +0000 UTC m=+17.226145174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mj97" (UniqueName: "kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97") pod "network-check-target-tl44d" (UID: "0b1c2c00-b265-42b8-b25c-28a4acd4f3c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:17:06.142596 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:06.141893 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:06.142596 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:06.142030 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:06.142596 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:06.142405 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:06.142596 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:06.142500 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:07.142974 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:07.142939 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:07.143415 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:07.143049 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:08.142717 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:08.142685 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:08.142864 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:08.142685 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:08.142864 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:08.142822 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:08.142864 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:08.142854 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:09.142046 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:09.141969 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:09.142402 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:09.142078 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:10.142864 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:10.142826 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:10.143345 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:10.142842 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:10.143345 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:10.142964 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:10.143345 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:10.143046 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:11.142760 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:11.142727 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:11.142966 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:11.142850 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:12.142504 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:12.142467 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:12.142504 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:12.142509 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:12.142736 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:12.142619 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:12.142782 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:12.142738 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:13.142699 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:13.142653 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:13.143097 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.142777 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:13.817445 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:13.817399 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:13.817445 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:13.817439 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:13.817716 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.817563 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 14:17:13.817716 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.817597 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:17:13.817716 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.817642 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret podName:9b27844f-7fba-4449-b514-1f2d9424f97e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:29.817622934 +0000 UTC m=+33.184884960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret") pod "global-pull-secret-syncer-np467" (UID: "9b27844f-7fba-4449-b514-1f2d9424f97e") : object "kube-system"/"original-pull-secret" not registered Jul 27 14:17:13.817716 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.817679 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs podName:a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de nodeName:}" failed. No retries permitted until 2026-07-27 14:17:29.817652312 +0000 UTC m=+33.184914337 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs") pod "network-metrics-daemon-lxpdl" (UID: "a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:17:13.918653 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:13.918569 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:13.918842 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.918721 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 14:17:13.918842 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.918740 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 14:17:13.918842 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.918752 2555 projected.go:196] Error preparing data for projected volume kube-api-access-2mj97 for pod openshift-network-diagnostics/network-check-target-tl44d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:17:13.918842 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:13.918811 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97 podName:0b1c2c00-b265-42b8-b25c-28a4acd4f3c7 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:29.918793867 +0000 UTC m=+33.286055910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mj97" (UniqueName: "kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97") pod "network-check-target-tl44d" (UID: "0b1c2c00-b265-42b8-b25c-28a4acd4f3c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:17:14.142814 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:14.142737 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:14.143225 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:14.142740 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:14.143225 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:14.142881 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:14.143225 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:14.142963 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:15.142032 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:15.141994 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:15.142296 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:15.142137 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:16.142510 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:16.142480 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:16.142971 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:16.142481 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:16.142971 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:16.142617 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:16.142971 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:16.142690 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:17.142978 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:17.142942 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:17.143421 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:17.143049 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:18.142729 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.142706 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:18.142861 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:18.142840 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:18.142970 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.142949 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:18.143081 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:18.143059 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:18.428372 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.427626 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k696n" event={"ID":"4b2d36d1-a3d2-477c-b22a-d7963daf4fb1","Type":"ContainerStarted","Data":"9d5639698c2e42f870935719dac8a8c53f983cda122b7c02ac201272e139d6fa"} Jul 27 14:17:18.432140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.432089 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gpfpw" event={"ID":"def06289-55b1-475f-a338-42014932d174","Type":"ContainerStarted","Data":"4eecac3c87f970e5cccbb4f53edb64a106f776b72ae9205645bd83f93551b265"} Jul 27 14:17:18.438367 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.438232 2555 generic.go:356] "Generic (PLEG): container finished" podID="77040d035aff638adc39fd418cfe0d87" containerID="5211ca934cd4aa3d0c282bd8162db72a6a9024ba6921ba682ccfe39f338703d9" exitCode=0 Jul 27 14:17:18.438367 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.438323 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" event={"ID":"77040d035aff638adc39fd418cfe0d87","Type":"ContainerDied","Data":"5211ca934cd4aa3d0c282bd8162db72a6a9024ba6921ba682ccfe39f338703d9"} Jul 27 14:17:18.445863 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.445737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" event={"ID":"91823a60abd6f5026e7827f004ecd18b","Type":"ContainerStarted","Data":"f3e00951aa00ed54227adf27211b33963f3aadfc0f3cac9da2f900a4ef8431cf"} Jul 27 14:17:18.460474 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.460162 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k696n" podStartSLOduration=2.368183293 podStartE2EDuration="21.460141681s" totalImagesPullingTime="19.091958388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:18.442329296 +0000 UTC m=+21.809591345" watchObservedRunningTime="2026-07-27 14:17:18.460141681 +0000 UTC m=+21.827403731" Jul 27 14:17:18.460474 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.460404 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" podStartSLOduration=2.343874787 podStartE2EDuration="21.460392736s" totalImagesPullingTime="19.116517949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:18.460267993 +0000 UTC m=+21.827530042" watchObservedRunningTime="2026-07-27 14:17:18.460392736 +0000 UTC m=+21.827654799" Jul 27 14:17:18.479025 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:18.478973 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gpfpw" podStartSLOduration=2.348913814 podStartE2EDuration="21.478951901s" totalImagesPullingTime="19.130038087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:18.477707178 +0000 UTC m=+21.844969228" watchObservedRunningTime="2026-07-27 14:17:18.478951901 +0000 UTC m=+21.846213950" Jul 27 14:17:19.142690 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.142364 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:19.142867 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:19.142733 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:19.449460 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.449428 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fvnrd" event={"ID":"eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3","Type":"ContainerStarted","Data":"8a06be303c21e4a95191d215cf70ed1f7addf87cfde32fa53476ee891fd19133"} Jul 27 14:17:19.451175 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.451152 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" event={"ID":"77040d035aff638adc39fd418cfe0d87","Type":"ContainerStarted","Data":"2e11474d552da6c9e5771188d76b9ace5a6a49099e4c0c895dada400d6c9b5e3"} Jul 27 14:17:19.453670 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.453628 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"433f2e25a7979044561765e5310ed44dd6830ffae243894b296b65183ece6772"} Jul 27 14:17:19.453768 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.453725 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"8c55acdc346da08f2818a6855a59a87b0bec09ce9348063526f767e85da635cf"} Jul 27 14:17:19.453834 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.453778 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"d14d28f457aacf453464fbb59cd933527beabe4b3a0f2381a3a153af5d1ef34e"} Jul 27 14:17:19.453834 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.453826 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"5bef8c3591fab6b7ed24932fb070534ee61d78e18fa468ddf43f8cbc27ee2c78"} Jul 27 14:17:19.453938 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.453874 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"1230a1fe8207f046a9d48dbc74b7bc996d9195e374603e78806fd979a49cf290"} Jul 27 14:17:19.453938 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.453921 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"eb3240f867f067a64d8a0b63154d1ff75ce16a0cfe6a07b4ffb8b97dba565c5e"} Jul 27 14:17:19.455106 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.455084 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-crs6s" event={"ID":"7272bcef-4347-40ac-9ce4-9de68958d689","Type":"ContainerStarted","Data":"77d6f8f34a49237ff5d0b5237f05e86db49aea393c41caddc92e09518b7e3ac5"} Jul 27 14:17:19.456356 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.456337 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" event={"ID":"c93e7038-cc5e-47f1-a26e-3d50a7eb963e","Type":"ContainerStarted","Data":"9b3ff982b4d123a26dc334f7ef51e1b5e20efde25836aae9935bc29669ee7065"} Jul 27 14:17:19.457621 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.457593 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jbjwv" event={"ID":"4bdecfbd-3e2f-47f0-a85b-2efffb7aef68","Type":"ContainerStarted","Data":"d493cdd619de06b5376703e15dee3bfacf72d1e33c2d052b236b4f8b153c18ae"} Jul 27 14:17:19.458932 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.458913 2555 generic.go:356] "Generic (PLEG): container finished" podID="acf16aa1-2fc6-4275-9845-a26701408128" containerID="b908c32beb349d395c5706175e8bc6520fb045d5c242b461fae5201eaabb648a" exitCode=0 Jul 27 14:17:19.459026 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.458970 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerDied","Data":"b908c32beb349d395c5706175e8bc6520fb045d5c242b461fae5201eaabb648a"} Jul 27 14:17:19.460387 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.460340 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nqrnp" event={"ID":"d78abc26-4e7e-4509-befb-3e19f7dc9b3e","Type":"ContainerStarted","Data":"d0f72ba960b19c96d5c79e87dcce790a5640ff1b8d7acbc86aa5c4000a4e5514"} Jul 27 14:17:19.468482 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.468447 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-223.ec2.internal" podStartSLOduration=21.468435794 podStartE2EDuration="21.468435794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:18.521829957 +0000 UTC m=+21.889092006" watchObservedRunningTime="2026-07-27 14:17:19.468435794 +0000 UTC m=+22.835697842" Jul 27 14:17:19.468607 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.468590 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-223.ec2.internal" podStartSLOduration=21.468585519 podStartE2EDuration="21.468585519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:19.468242574 +0000 UTC m=+22.835504623" watchObservedRunningTime="2026-07-27 14:17:19.468585519 +0000 UTC m=+22.835847567" Jul 27 14:17:19.484700 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.484646 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jbjwv" podStartSLOduration=3.350585618 podStartE2EDuration="22.484634716s" totalImagesPullingTime="19.134049098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:19.484387324 +0000 UTC m=+22.851649374" watchObservedRunningTime="2026-07-27 14:17:19.484634716 +0000 UTC m=+22.851896763" Jul 27 14:17:19.523446 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.523397 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nqrnp" podStartSLOduration=3.433133862 podStartE2EDuration="22.523383147s" totalImagesPullingTime="19.090249285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:19.499712367 +0000 UTC m=+22.866974411" watchObservedRunningTime="2026-07-27 14:17:19.523383147 +0000 UTC m=+22.890645228" Jul 27 14:17:19.729271 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:19.729235 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 27 14:17:20.142486 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:20.142398 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:20.142486 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:20.142443 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:20.142792 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:20.142548 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:20.142792 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:20.142678 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:20.193676 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:20.193524 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T14:17:19.729254586Z","UUID":"1f0973b4-72c0-4b54-91d2-5fccf6a4cb0b","Handler":null,"Name":"","Endpoint":""} Jul 27 14:17:20.195100 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:20.195068 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 27 14:17:20.195232 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:20.195108 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 27 14:17:20.465454 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:20.465375 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" event={"ID":"c93e7038-cc5e-47f1-a26e-3d50a7eb963e","Type":"ContainerStarted","Data":"20e2abc85d94d05e9db299a91a035daf94386a7b4556c3422e6319133dc9f1df"} Jul 27 14:17:21.142508 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:21.142307 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:21.142727 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:21.142588 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:21.472035 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:21.471982 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"40f07ba0f6a7748285320cbcca06adec2715cfbe47b49036ae3cadfb596b0946"} Jul 27 14:17:21.474442 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:21.474417 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" event={"ID":"c93e7038-cc5e-47f1-a26e-3d50a7eb963e","Type":"ContainerStarted","Data":"a40fee36d54363a14fb1056eab684a9ffda5ee99bf868ecf34aff773ecfe04c3"} Jul 27 14:17:21.493404 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:21.493362 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6l49" podStartSLOduration=3.030595519 podStartE2EDuration="24.493349799s" totalImagesPullingTime="21.46275428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:21.493108188 +0000 UTC m=+24.860370236" watchObservedRunningTime="2026-07-27 14:17:21.493349799 +0000 UTC m=+24.860611847" Jul 27 14:17:21.493794 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:21.493772 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-crs6s" podStartSLOduration=5.3836897409999995 podStartE2EDuration="24.493764272s" totalImagesPullingTime="19.110074531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:19.542915112 +0000 UTC m=+22.910177159" watchObservedRunningTime="2026-07-27 14:17:21.493764272 +0000 UTC m=+24.861026319" Jul 27 14:17:21.964856 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:21.964819 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k696n" Jul 27 14:17:21.965517 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:21.965490 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k696n" Jul 27 14:17:22.142692 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:22.142641 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:22.142868 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:22.142639 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:22.142868 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:22.142771 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:22.142986 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:22.142914 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:22.962066 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:22.962031 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k696n" Jul 27 14:17:22.962804 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:22.962774 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k696n" Jul 27 14:17:23.142769 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:23.142742 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:23.142916 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:23.142878 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:24.141867 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:24.141841 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:24.142309 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:24.141941 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:24.142309 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:24.141994 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:24.142309 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:24.142079 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:24.486194 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:24.485984 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" event={"ID":"38423966-1ba9-464f-8227-256adcb6b870","Type":"ContainerStarted","Data":"886e82217576a099cf66e300fa5f280914d9b6275381bb3463dc5a980f6f938d"} Jul 27 14:17:24.486327 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:24.486285 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:17:24.486373 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:24.486331 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:17:24.488087 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:24.488059 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerStarted","Data":"edbe80fc660acbfb12984e919d4f7ee7472c3eca9636d7a4160274328cc64e5a"} Jul 27 14:17:24.501944 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:24.501921 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:17:24.512245 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:24.512205 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" podStartSLOduration=6.574836192 podStartE2EDuration="26.512192643s" totalImagesPullingTime="19.937356451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:24.511875052 +0000 UTC m=+27.879137100" watchObservedRunningTime="2026-07-27 14:17:24.512192643 +0000 UTC m=+27.879454691" Jul 27 14:17:25.142448 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:25.142418 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:25.142915 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:25.142519 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:25.493886 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:25.493846 2555 generic.go:356] "Generic (PLEG): container finished" podID="acf16aa1-2fc6-4275-9845-a26701408128" containerID="edbe80fc660acbfb12984e919d4f7ee7472c3eca9636d7a4160274328cc64e5a" exitCode=0 Jul 27 14:17:25.495222 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:25.495190 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerDied","Data":"edbe80fc660acbfb12984e919d4f7ee7472c3eca9636d7a4160274328cc64e5a"} Jul 27 14:17:25.495555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:25.495535 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:17:25.515840 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:25.515812 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:17:26.142770 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:26.142736 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:26.143170 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:26.142740 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:26.143170 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:26.142872 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:26.143170 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:26.142955 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:26.515285 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:26.515249 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lxpdl"] Jul 27 14:17:26.515455 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:26.515339 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:26.515455 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:26.515427 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:26.517720 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:26.517687 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tl44d"] Jul 27 14:17:26.517864 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:26.517786 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:26.517926 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:26.517859 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:26.520474 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:26.520452 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-np467"] Jul 27 14:17:26.520570 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:26.520519 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:26.520613 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:26.520580 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:27.500934 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:27.500694 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerStarted","Data":"11ecbe9579197a3a705ca3ecb2bde70ce5da2c0fff46cbcf04503be656191fd6"} Jul 27 14:17:28.142360 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:28.142320 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:28.142534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:28.142326 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:28.142534 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:28.142443 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:28.142534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:28.142326 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:28.142742 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:28.142518 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:28.142742 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:28.142582 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:28.505124 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:28.505090 2555 generic.go:356] "Generic (PLEG): container finished" podID="acf16aa1-2fc6-4275-9845-a26701408128" containerID="11ecbe9579197a3a705ca3ecb2bde70ce5da2c0fff46cbcf04503be656191fd6" exitCode=0 Jul 27 14:17:28.505634 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:28.505133 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerDied","Data":"11ecbe9579197a3a705ca3ecb2bde70ce5da2c0fff46cbcf04503be656191fd6"} Jul 27 14:17:29.834017 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:29.833987 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:29.834017 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:29.834022 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:29.834542 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:29.834136 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 14:17:29.834542 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:29.834195 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret podName:9b27844f-7fba-4449-b514-1f2d9424f97e nodeName:}" failed. No retries permitted until 2026-07-27 14:18:01.834181334 +0000 UTC m=+65.201443374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret") pod "global-pull-secret-syncer-np467" (UID: "9b27844f-7fba-4449-b514-1f2d9424f97e") : object "kube-system"/"original-pull-secret" not registered Jul 27 14:17:29.834542 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:29.834137 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:17:29.834542 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:29.834270 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs podName:a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de nodeName:}" failed. No retries permitted until 2026-07-27 14:18:01.834255772 +0000 UTC m=+65.201517798 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs") pod "network-metrics-daemon-lxpdl" (UID: "a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 14:17:29.935209 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:29.935180 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:29.935378 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:29.935297 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 14:17:29.935378 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:29.935310 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 14:17:29.935378 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:29.935319 2555 projected.go:196] Error preparing data for projected volume kube-api-access-2mj97 for pod openshift-network-diagnostics/network-check-target-tl44d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:17:29.935378 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:29.935365 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97 podName:0b1c2c00-b265-42b8-b25c-28a4acd4f3c7 nodeName:}" failed. No retries permitted until 2026-07-27 14:18:01.935353208 +0000 UTC m=+65.302615247 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mj97" (UniqueName: "kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97") pod "network-check-target-tl44d" (UID: "0b1c2c00-b265-42b8-b25c-28a4acd4f3c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 14:17:30.142243 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:30.142205 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:30.142243 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:30.142226 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:30.142243 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:30.142231 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:30.142535 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:30.142326 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl44d" podUID="0b1c2c00-b265-42b8-b25c-28a4acd4f3c7" Jul 27 14:17:30.142535 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:30.142374 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-np467" podUID="9b27844f-7fba-4449-b514-1f2d9424f97e" Jul 27 14:17:30.142535 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:30.142454 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lxpdl" podUID="a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de" Jul 27 14:17:30.511351 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:30.511321 2555 generic.go:356] "Generic (PLEG): container finished" podID="acf16aa1-2fc6-4275-9845-a26701408128" containerID="eae5553161234da91befd1ddafb7c07e3d99e7fcc547f5781b9476d6c69fe05e" exitCode=0 Jul 27 14:17:30.511529 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:30.511384 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerDied","Data":"eae5553161234da91befd1ddafb7c07e3d99e7fcc547f5781b9476d6c69fe05e"} Jul 27 14:17:31.024082 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.023999 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-223.ec2.internal" event="NodeReady" Jul 27 14:17:31.024524 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.024167 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 27 14:17:31.060103 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.060070 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76f6585d46-wlhhh"] Jul 27 14:17:31.064159 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.064134 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.066780 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.066761 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 27 14:17:31.067261 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.067236 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 27 14:17:31.067261 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.067258 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qrv8d\"" Jul 27 14:17:31.067412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.067357 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 27 14:17:31.073597 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.073566 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 27 14:17:31.082889 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.082865 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f6585d46-wlhhh"] Jul 27 14:17:31.083777 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.083728 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8cbvq"] Jul 27 14:17:31.087753 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.087728 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bwz8r"] Jul 27 14:17:31.087860 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.087812 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.090462 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.090445 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 27 14:17:31.090814 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.090795 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9kwzw\"" Jul 27 14:17:31.090913 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.090838 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 27 14:17:31.091284 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.091268 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bwz8r" Jul 27 14:17:31.094641 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.094619 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 27 14:17:31.094755 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.094700 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 27 14:17:31.094755 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.094742 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 27 14:17:31.094867 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.094761 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fj826\"" Jul 27 14:17:31.098258 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.098234 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8cbvq"] Jul 27 14:17:31.102222 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.102199 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bwz8r"] Jul 27 14:17:31.143364 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.143327 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.143364 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.143367 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3516fbc8-37ce-4433-bbff-4faea6399efe-ca-trust-extracted\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.143588 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.143425 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcdtm\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-kube-api-access-dcdtm\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.143648 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.143602 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-trusted-ca\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.143648 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.143632 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-bound-sa-token\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.143772 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.143679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-certificates\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.143772 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.143715 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-image-registry-private-configuration\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.143772 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.143737 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-installation-pull-secrets\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.244761 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.244727 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zgqk\" (UniqueName: \"kubernetes.io/projected/f0dfb067-bcb2-4d4c-88be-f93bec24022f-kube-api-access-9zgqk\") pod \"ingress-canary-bwz8r\" (UID: \"f0dfb067-bcb2-4d4c-88be-f93bec24022f\") " pod="openshift-ingress-canary/ingress-canary-bwz8r" Jul 27 14:17:31.244951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.244783 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.244951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.244842 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3516fbc8-37ce-4433-bbff-4faea6399efe-ca-trust-extracted\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.244951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.244883 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcdtm\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-kube-api-access-dcdtm\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.244951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.244907 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0dfb067-bcb2-4d4c-88be-f93bec24022f-cert\") pod \"ingress-canary-bwz8r\" (UID: \"f0dfb067-bcb2-4d4c-88be-f93bec24022f\") " pod="openshift-ingress-canary/ingress-canary-bwz8r" Jul 27 14:17:31.244951 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.244910 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 14:17:31.244951 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.244925 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f6585d46-wlhhh: secret "image-registry-tls" not found Jul 27 14:17:31.245244 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.244976 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls podName:3516fbc8-37ce-4433-bbff-4faea6399efe nodeName:}" failed. No retries permitted until 2026-07-27 14:17:31.744960675 +0000 UTC m=+35.112222718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls") pod "image-registry-76f6585d46-wlhhh" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe") : secret "image-registry-tls" not found Jul 27 14:17:31.245244 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245053 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-trusted-ca\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.245244 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245085 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/00a47fc4-008e-4b66-8ae2-c5fed963599a-config-volume\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.245244 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245115 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-bound-sa-token\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.245244 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245139 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.245244 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245162 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a47fc4-008e-4b66-8ae2-c5fed963599a-tmp-dir\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.245244 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245196 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-certificates\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.245244 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245226 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-image-registry-private-configuration\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.245963 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245922 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfwmv\" (UniqueName: \"kubernetes.io/projected/00a47fc4-008e-4b66-8ae2-c5fed963599a-kube-api-access-mfwmv\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.246086 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245975 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3516fbc8-37ce-4433-bbff-4faea6399efe-ca-trust-extracted\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.246086 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.245999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-installation-pull-secrets\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.246512 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.246493 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-certificates\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.246692 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.246671 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-trusted-ca\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.251621 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.251597 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-image-registry-private-configuration\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.251744 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.251627 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-installation-pull-secrets\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.254004 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.253980 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcdtm\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-kube-api-access-dcdtm\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.254324 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.254305 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-bound-sa-token\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.346640 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.346547 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.346640 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.346597 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a47fc4-008e-4b66-8ae2-c5fed963599a-tmp-dir\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.346888 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.346641 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfwmv\" (UniqueName: \"kubernetes.io/projected/00a47fc4-008e-4b66-8ae2-c5fed963599a-kube-api-access-mfwmv\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.346888 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.346718 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zgqk\" (UniqueName: \"kubernetes.io/projected/f0dfb067-bcb2-4d4c-88be-f93bec24022f-kube-api-access-9zgqk\") pod \"ingress-canary-bwz8r\" (UID: \"f0dfb067-bcb2-4d4c-88be-f93bec24022f\") " pod="openshift-ingress-canary/ingress-canary-bwz8r" Jul 27 14:17:31.346888 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.346729 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 14:17:31.346888 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.346762 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0dfb067-bcb2-4d4c-88be-f93bec24022f-cert\") pod \"ingress-canary-bwz8r\" (UID: \"f0dfb067-bcb2-4d4c-88be-f93bec24022f\") " pod="openshift-ingress-canary/ingress-canary-bwz8r" Jul 27 14:17:31.346888 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.346805 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls podName:00a47fc4-008e-4b66-8ae2-c5fed963599a nodeName:}" failed. No retries permitted until 2026-07-27 14:17:31.846782319 +0000 UTC m=+35.214044364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls") pod "dns-default-8cbvq" (UID: "00a47fc4-008e-4b66-8ae2-c5fed963599a") : secret "dns-default-metrics-tls" not found Jul 27 14:17:31.346888 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.346868 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/00a47fc4-008e-4b66-8ae2-c5fed963599a-config-volume\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.347168 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.347010 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a47fc4-008e-4b66-8ae2-c5fed963599a-tmp-dir\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.347419 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.347394 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/00a47fc4-008e-4b66-8ae2-c5fed963599a-config-volume\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.349488 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.349466 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0dfb067-bcb2-4d4c-88be-f93bec24022f-cert\") pod \"ingress-canary-bwz8r\" (UID: \"f0dfb067-bcb2-4d4c-88be-f93bec24022f\") " pod="openshift-ingress-canary/ingress-canary-bwz8r" Jul 27 14:17:31.358443 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.358420 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfwmv\" (UniqueName: \"kubernetes.io/projected/00a47fc4-008e-4b66-8ae2-c5fed963599a-kube-api-access-mfwmv\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.358744 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.358724 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zgqk\" (UniqueName: \"kubernetes.io/projected/f0dfb067-bcb2-4d4c-88be-f93bec24022f-kube-api-access-9zgqk\") pod \"ingress-canary-bwz8r\" (UID: \"f0dfb067-bcb2-4d4c-88be-f93bec24022f\") " pod="openshift-ingress-canary/ingress-canary-bwz8r" Jul 27 14:17:31.406867 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.406832 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bwz8r" Jul 27 14:17:31.556187 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.555987 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bwz8r"] Jul 27 14:17:31.559150 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:17:31.559125 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0dfb067_bcb2_4d4c_88be_f93bec24022f.slice/crio-8a94ba88a3afcc8d11b6e8de918c13594069c051f013b9cc2a90c8108598e1f8 WatchSource:0}: Error finding container 8a94ba88a3afcc8d11b6e8de918c13594069c051f013b9cc2a90c8108598e1f8: Status 404 returned error can't find the container with id 8a94ba88a3afcc8d11b6e8de918c13594069c051f013b9cc2a90c8108598e1f8 Jul 27 14:17:31.750358 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.750322 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:31.750542 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.750486 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 14:17:31.750542 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.750509 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f6585d46-wlhhh: secret "image-registry-tls" not found Jul 27 14:17:31.750626 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.750572 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls podName:3516fbc8-37ce-4433-bbff-4faea6399efe nodeName:}" failed. No retries permitted until 2026-07-27 14:17:32.75055289 +0000 UTC m=+36.117814929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls") pod "image-registry-76f6585d46-wlhhh" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe") : secret "image-registry-tls" not found Jul 27 14:17:31.851807 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:31.851769 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:31.851982 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.851883 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 14:17:31.851982 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:31.851944 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls podName:00a47fc4-008e-4b66-8ae2-c5fed963599a nodeName:}" failed. No retries permitted until 2026-07-27 14:17:32.851926931 +0000 UTC m=+36.219188976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls") pod "dns-default-8cbvq" (UID: "00a47fc4-008e-4b66-8ae2-c5fed963599a") : secret "dns-default-metrics-tls" not found Jul 27 14:17:32.142877 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.142789 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:17:32.143541 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.142790 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:17:32.143541 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.142790 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:17:32.146545 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.146525 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 27 14:17:32.146702 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.146597 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mmpms\"" Jul 27 14:17:32.146702 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.146633 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 27 14:17:32.147570 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.147551 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nn5ms\"" Jul 27 14:17:32.147680 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.147580 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 14:17:32.147831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.147810 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 14:17:32.518318 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.518285 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bwz8r" event={"ID":"f0dfb067-bcb2-4d4c-88be-f93bec24022f","Type":"ContainerStarted","Data":"8a94ba88a3afcc8d11b6e8de918c13594069c051f013b9cc2a90c8108598e1f8"} Jul 27 14:17:32.758689 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.758641 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:32.758865 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:32.758816 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 14:17:32.758865 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:32.758839 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f6585d46-wlhhh: secret "image-registry-tls" not found Jul 27 14:17:32.758989 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:32.758907 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls podName:3516fbc8-37ce-4433-bbff-4faea6399efe nodeName:}" failed. No retries permitted until 2026-07-27 14:17:34.758886724 +0000 UTC m=+38.126148763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls") pod "image-registry-76f6585d46-wlhhh" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe") : secret "image-registry-tls" not found Jul 27 14:17:32.859917 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:32.859827 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:32.860090 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:32.859985 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 14:17:32.860090 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:32.860057 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls podName:00a47fc4-008e-4b66-8ae2-c5fed963599a nodeName:}" failed. No retries permitted until 2026-07-27 14:17:34.860035969 +0000 UTC m=+38.227297998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls") pod "dns-default-8cbvq" (UID: "00a47fc4-008e-4b66-8ae2-c5fed963599a") : secret "dns-default-metrics-tls" not found Jul 27 14:17:34.775273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:34.775231 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:34.775717 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:34.775402 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 14:17:34.775717 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:34.775418 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f6585d46-wlhhh: secret "image-registry-tls" not found Jul 27 14:17:34.775717 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:34.775475 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls podName:3516fbc8-37ce-4433-bbff-4faea6399efe nodeName:}" failed. No retries permitted until 2026-07-27 14:17:38.775457058 +0000 UTC m=+42.142719104 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls") pod "image-registry-76f6585d46-wlhhh" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe") : secret "image-registry-tls" not found Jul 27 14:17:34.876310 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:34.876270 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:34.876503 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:34.876440 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 14:17:34.876559 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:34.876512 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls podName:00a47fc4-008e-4b66-8ae2-c5fed963599a nodeName:}" failed. No retries permitted until 2026-07-27 14:17:38.876490978 +0000 UTC m=+42.243753018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls") pod "dns-default-8cbvq" (UID: "00a47fc4-008e-4b66-8ae2-c5fed963599a") : secret "dns-default-metrics-tls" not found Jul 27 14:17:36.530401 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:36.530368 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bwz8r" event={"ID":"f0dfb067-bcb2-4d4c-88be-f93bec24022f","Type":"ContainerStarted","Data":"9cd681ca53c06212d9d677be36d85635bc4467720ee53a6989e49ecea820acd9"} Jul 27 14:17:36.533051 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:36.533008 2555 generic.go:356] "Generic (PLEG): container finished" podID="acf16aa1-2fc6-4275-9845-a26701408128" containerID="3dc71debcdd1bcc5f89307d52442bf8517025e08ff1b7d22a728f7c0332129e1" exitCode=0 Jul 27 14:17:36.533139 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:36.533060 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerDied","Data":"3dc71debcdd1bcc5f89307d52442bf8517025e08ff1b7d22a728f7c0332129e1"} Jul 27 14:17:36.548817 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:36.548772 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bwz8r" podStartSLOduration=1.621009258 podStartE2EDuration="5.548757705s" totalImagesPullingTime="3.927748447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:36.548315715 +0000 UTC m=+39.915577763" watchObservedRunningTime="2026-07-27 14:17:36.548757705 +0000 UTC m=+39.916019731" Jul 27 14:17:37.213069 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.213038 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7"] Jul 27 14:17:37.239223 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.239190 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs"] Jul 27 14:17:37.239385 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.239357 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.242938 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.242916 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 27 14:17:37.243092 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.243030 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vsv7l\"" Jul 27 14:17:37.243266 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.243248 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 27 14:17:37.243611 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.243589 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 27 14:17:37.245883 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.245859 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 27 14:17:37.255243 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.255222 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s"] Jul 27 14:17:37.255336 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.255260 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.258029 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.258006 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 27 14:17:37.258123 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.258048 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-q6wg6\"" Jul 27 14:17:37.258724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.258706 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 27 14:17:37.258808 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.258740 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 27 14:17:37.258878 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.258851 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 27 14:17:37.273280 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.273256 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j"] Jul 27 14:17:37.273385 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.273362 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.276676 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.276647 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 27 14:17:37.279336 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.279320 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 27 14:17:37.280884 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.280870 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 27 14:17:37.281416 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.281400 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-dxnzt\"" Jul 27 14:17:37.281471 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.281440 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 27 14:17:37.295478 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.295450 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-nhx74"] Jul 27 14:17:37.295572 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.295528 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:37.302404 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.302384 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ccjxz\"" Jul 27 14:17:37.302493 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.302432 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 27 14:17:37.302567 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.302490 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 27 14:17:37.302567 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.302553 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 27 14:17:37.321603 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.321576 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7"] Jul 27 14:17:37.321603 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.321609 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs"] Jul 27 14:17:37.321761 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.321624 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j"] Jul 27 14:17:37.321761 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.321673 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s"] Jul 27 14:17:37.321761 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.321691 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-nhx74"] Jul 27 14:17:37.321847 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.321768 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.327560 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.327539 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 27 14:17:37.327922 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.327901 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 27 14:17:37.328010 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.327908 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 27 14:17:37.328010 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.327969 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-q8fxf\"" Jul 27 14:17:37.335184 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.335164 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 27 14:17:37.337782 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.337759 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 27 14:17:37.364334 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.364300 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd"] Jul 27 14:17:37.377606 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.377575 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7745b5dc69-b8hmd"] Jul 27 14:17:37.377780 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.377742 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd" Jul 27 14:17:37.380586 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.380564 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8b5g9\"" Jul 27 14:17:37.380586 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.380587 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 27 14:17:37.380768 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.380607 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 27 14:17:37.387736 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.387715 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wdpj7"] Jul 27 14:17:37.387859 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.387848 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.394156 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.394134 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vjx9h\"" Jul 27 14:17:37.394410 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.394391 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 27 14:17:37.394410 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.394407 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 27 14:17:37.394538 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.394425 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 27 14:17:37.394538 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.394407 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 27 14:17:37.395297 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.395281 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 27 14:17:37.396233 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396209 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k69gm\" (UniqueName: \"kubernetes.io/projected/a9ba173c-70f2-4c02-a10d-f48753f7f720-kube-api-access-k69gm\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.396321 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396235 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 27 14:17:37.396365 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396323 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:37.396398 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396367 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/028ffee0-b008-47b4-93b0-996dfbd4f7b1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.396436 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tznxj\" (UniqueName: \"kubernetes.io/projected/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-kube-api-access-tznxj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.396436 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396424 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq4fz\" (UniqueName: \"kubernetes.io/projected/878dade8-dbdf-4710-9b30-b3470dcb4221-kube-api-access-kq4fz\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:37.396512 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396481 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.396555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396508 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmq9t\" (UniqueName: \"kubernetes.io/projected/028ffee0-b008-47b4-93b0-996dfbd4f7b1-kube-api-access-dmq9t\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.396555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396538 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.396625 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396568 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.396625 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396602 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9ba173c-70f2-4c02-a10d-f48753f7f720-config\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.396749 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.396692 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9ba173c-70f2-4c02-a10d-f48753f7f720-serving-cert\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.401804 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.401785 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd"] Jul 27 14:17:37.401893 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.401814 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wdpj7"] Jul 27 14:17:37.401893 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.401889 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.404783 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.404766 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 27 14:17:37.405324 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.405297 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 27 14:17:37.405394 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.405383 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 27 14:17:37.410524 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.410507 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 27 14:17:37.411182 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.411160 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dhmkg\"" Jul 27 14:17:37.416001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.415976 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 27 14:17:37.418456 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.418431 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7745b5dc69-b8hmd"] Jul 27 14:17:37.435600 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.435569 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw"] Jul 27 14:17:37.449915 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.449890 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw" Jul 27 14:17:37.451259 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.451236 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw"] Jul 27 14:17:37.452820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.452801 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ftcml\"" Jul 27 14:17:37.498122 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.497984 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tznxj\" (UniqueName: \"kubernetes.io/projected/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-kube-api-access-tznxj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.498122 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498028 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/db5170c1-44f1-4ee2-a373-78788f49e622-trusted-ca\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.498122 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498086 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cdl2\" (UniqueName: \"kubernetes.io/projected/7678c3a4-0414-4678-ade4-767be3864a91-kube-api-access-4cdl2\") pod \"volume-data-source-validator-549c6d54cf-z4nfd\" (UID: \"7678c3a4-0414-4678-ade4-767be3864a91\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd" Jul 27 14:17:37.498122 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498122 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.498439 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498146 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fb3e7b-bef0-473d-a98e-0cbac98c891c-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.498439 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498174 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k69gm\" (UniqueName: \"kubernetes.io/projected/a9ba173c-70f2-4c02-a10d-f48753f7f720-kube-api-access-k69gm\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.498439 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498248 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-stats-auth\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.498439 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498409 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/028ffee0-b008-47b4-93b0-996dfbd4f7b1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.498803 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498461 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbhcx\" (UniqueName: \"kubernetes.io/projected/db5170c1-44f1-4ee2-a373-78788f49e622-kube-api-access-lbhcx\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.498803 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498489 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9ba173c-70f2-4c02-a10d-f48753f7f720-serving-cert\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.498803 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498528 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76fb3e7b-bef0-473d-a98e-0cbac98c891c-tmp\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.498803 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498713 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:37.498803 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498737 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq4fz\" (UniqueName: \"kubernetes.io/projected/878dade8-dbdf-4710-9b30-b3470dcb4221-kube-api-access-kq4fz\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:37.498803 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.499048 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498806 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmq9t\" (UniqueName: \"kubernetes.io/projected/028ffee0-b008-47b4-93b0-996dfbd4f7b1-kube-api-access-dmq9t\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.499048 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.498831 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.499048 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:37.498839 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 14:17:37.499048 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:37.498897 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls podName:878dade8-dbdf-4710-9b30-b3470dcb4221 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:37.998877372 +0000 UTC m=+41.366139412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls") pod "cluster-samples-operator-6748489656-nhl9j" (UID: "878dade8-dbdf-4710-9b30-b3470dcb4221") : secret "samples-operator-tls" not found Jul 27 14:17:37.499371 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499349 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.499442 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499366 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.499442 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499400 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/028ffee0-b008-47b4-93b0-996dfbd4f7b1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.499442 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499420 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-default-certificate\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.499442 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499442 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fb3e7b-bef0-473d-a98e-0cbac98c891c-service-ca-bundle\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.499590 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499465 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db5170c1-44f1-4ee2-a373-78788f49e622-config\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.499590 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:37.499496 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:37.499590 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499498 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.499590 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499523 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xl9h\" (UniqueName: \"kubernetes.io/projected/c5d14ecc-98e1-45cf-9071-0106bec58113-kube-api-access-6xl9h\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.499590 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499540 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/76fb3e7b-bef0-473d-a98e-0cbac98c891c-snapshots\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.499590 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:37.499555 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls podName:028ffee0-b008-47b4-93b0-996dfbd4f7b1 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:37.999539468 +0000 UTC m=+41.366801496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fwmw7" (UID: "028ffee0-b008-47b4-93b0-996dfbd4f7b1") : secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:37.499846 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499604 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76fb3e7b-bef0-473d-a98e-0cbac98c891c-serving-cert\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.499846 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499625 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmxmw\" (UniqueName: \"kubernetes.io/projected/76fb3e7b-bef0-473d-a98e-0cbac98c891c-kube-api-access-jmxmw\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.499846 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499691 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9ba173c-70f2-4c02-a10d-f48753f7f720-config\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.499846 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.499710 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db5170c1-44f1-4ee2-a373-78788f49e622-serving-cert\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.500151 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.500135 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9ba173c-70f2-4c02-a10d-f48753f7f720-config\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.502307 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.502287 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9ba173c-70f2-4c02-a10d-f48753f7f720-serving-cert\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.502391 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.502307 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.512653 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.512630 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tznxj\" (UniqueName: \"kubernetes.io/projected/f86b19c8-ef8a-4e72-8048-c9fe8112d3ec-kube-api-access-tznxj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s\" (UID: \"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.512785 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.512713 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k69gm\" (UniqueName: \"kubernetes.io/projected/a9ba173c-70f2-4c02-a10d-f48753f7f720-kube-api-access-k69gm\") pod \"service-ca-operator-5d6c75f84f-q9dvs\" (UID: \"a9ba173c-70f2-4c02-a10d-f48753f7f720\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.515625 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.515606 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmq9t\" (UniqueName: \"kubernetes.io/projected/028ffee0-b008-47b4-93b0-996dfbd4f7b1-kube-api-access-dmq9t\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:37.515735 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.515706 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq4fz\" (UniqueName: \"kubernetes.io/projected/878dade8-dbdf-4710-9b30-b3470dcb4221-kube-api-access-kq4fz\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:37.538065 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.538036 2555 generic.go:356] "Generic (PLEG): container finished" podID="acf16aa1-2fc6-4275-9845-a26701408128" containerID="363b2c9657dbb296c96faf8c3bc1848f788977b20c556f9dc8108b0e7368e285" exitCode=0 Jul 27 14:17:37.538381 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.538117 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerDied","Data":"363b2c9657dbb296c96faf8c3bc1848f788977b20c556f9dc8108b0e7368e285"} Jul 27 14:17:37.566516 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.566491 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" Jul 27 14:17:37.581008 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.580987 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" Jul 27 14:17:37.600311 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.600282 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76fb3e7b-bef0-473d-a98e-0cbac98c891c-serving-cert\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.600433 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.600329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxmw\" (UniqueName: \"kubernetes.io/projected/76fb3e7b-bef0-473d-a98e-0cbac98c891c-kube-api-access-jmxmw\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.600433 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.600376 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db5170c1-44f1-4ee2-a373-78788f49e622-serving-cert\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.600546 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.600442 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/db5170c1-44f1-4ee2-a373-78788f49e622-trusted-ca\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.600546 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.600472 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cdl2\" (UniqueName: \"kubernetes.io/projected/7678c3a4-0414-4678-ade4-767be3864a91-kube-api-access-4cdl2\") pod \"volume-data-source-validator-549c6d54cf-z4nfd\" (UID: \"7678c3a4-0414-4678-ade4-767be3864a91\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd" Jul 27 14:17:37.601422 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.601396 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/db5170c1-44f1-4ee2-a373-78788f49e622-trusted-ca\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.601535 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.601504 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.601683 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:37.601652 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:38.101633586 +0000 UTC m=+41.468895632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : configmap references non-existent config key: service-ca.crt Jul 27 14:17:37.601896 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.601871 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fb3e7b-bef0-473d-a98e-0cbac98c891c-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.601976 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.601917 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7wkq\" (UniqueName: \"kubernetes.io/projected/723b1610-1571-4007-8a24-6451d3bf2153-kube-api-access-c7wkq\") pod \"network-check-source-7d88cb55d8-4fqqw\" (UID: \"723b1610-1571-4007-8a24-6451d3bf2153\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw" Jul 27 14:17:37.601976 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.601955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-stats-auth\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.602086 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.601998 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbhcx\" (UniqueName: \"kubernetes.io/projected/db5170c1-44f1-4ee2-a373-78788f49e622-kube-api-access-lbhcx\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.602086 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.602041 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76fb3e7b-bef0-473d-a98e-0cbac98c891c-tmp\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.602185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.602118 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-default-certificate\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.602185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.602142 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fb3e7b-bef0-473d-a98e-0cbac98c891c-service-ca-bundle\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.602185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.602169 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db5170c1-44f1-4ee2-a373-78788f49e622-config\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.602333 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.602195 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.602333 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.602224 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xl9h\" (UniqueName: \"kubernetes.io/projected/c5d14ecc-98e1-45cf-9071-0106bec58113-kube-api-access-6xl9h\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.602333 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.602250 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/76fb3e7b-bef0-473d-a98e-0cbac98c891c-snapshots\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.602936 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.602876 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76fb3e7b-bef0-473d-a98e-0cbac98c891c-tmp\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.603282 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.603258 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/76fb3e7b-bef0-473d-a98e-0cbac98c891c-snapshots\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.603634 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.603413 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db5170c1-44f1-4ee2-a373-78788f49e622-serving-cert\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.603634 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.603440 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fb3e7b-bef0-473d-a98e-0cbac98c891c-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.603634 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:37.603520 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 14:17:37.603634 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:37.603569 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:38.103552364 +0000 UTC m=+41.470814394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : secret "router-metrics-certs-default" not found Jul 27 14:17:37.603902 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.603766 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fb3e7b-bef0-473d-a98e-0cbac98c891c-service-ca-bundle\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.603902 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.603884 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db5170c1-44f1-4ee2-a373-78788f49e622-config\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.604080 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.604062 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76fb3e7b-bef0-473d-a98e-0cbac98c891c-serving-cert\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.605186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.605167 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-default-certificate\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.605302 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.605285 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-stats-auth\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.611187 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.611166 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmxmw\" (UniqueName: \"kubernetes.io/projected/76fb3e7b-bef0-473d-a98e-0cbac98c891c-kube-api-access-jmxmw\") pod \"insights-operator-5885fdb759-wdpj7\" (UID: \"76fb3e7b-bef0-473d-a98e-0cbac98c891c\") " pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.611309 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.611286 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cdl2\" (UniqueName: \"kubernetes.io/projected/7678c3a4-0414-4678-ade4-767be3864a91-kube-api-access-4cdl2\") pod \"volume-data-source-validator-549c6d54cf-z4nfd\" (UID: \"7678c3a4-0414-4678-ade4-767be3864a91\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd" Jul 27 14:17:37.612582 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.612565 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbhcx\" (UniqueName: \"kubernetes.io/projected/db5170c1-44f1-4ee2-a373-78788f49e622-kube-api-access-lbhcx\") pod \"console-operator-584dcb6b54-nhx74\" (UID: \"db5170c1-44f1-4ee2-a373-78788f49e622\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.617802 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.617779 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xl9h\" (UniqueName: \"kubernetes.io/projected/c5d14ecc-98e1-45cf-9071-0106bec58113-kube-api-access-6xl9h\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:37.629672 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.629635 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:37.686492 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.686465 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd" Jul 27 14:17:37.704076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.703720 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7wkq\" (UniqueName: \"kubernetes.io/projected/723b1610-1571-4007-8a24-6451d3bf2153-kube-api-access-c7wkq\") pod \"network-check-source-7d88cb55d8-4fqqw\" (UID: \"723b1610-1571-4007-8a24-6451d3bf2153\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw" Jul 27 14:17:37.710498 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.709797 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-wdpj7" Jul 27 14:17:37.713906 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.713858 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7wkq\" (UniqueName: \"kubernetes.io/projected/723b1610-1571-4007-8a24-6451d3bf2153-kube-api-access-c7wkq\") pod \"network-check-source-7d88cb55d8-4fqqw\" (UID: \"723b1610-1571-4007-8a24-6451d3bf2153\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw" Jul 27 14:17:37.743289 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.742546 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs"] Jul 27 14:17:37.747069 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:17:37.747014 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ba173c_70f2_4c02_a10d_f48753f7f720.slice/crio-5f17d60622572267cdf7c71fc563ac8e9c57bb934df01eb04f4648c1528b070a WatchSource:0}: Error finding container 5f17d60622572267cdf7c71fc563ac8e9c57bb934df01eb04f4648c1528b070a: Status 404 returned error can't find the container with id 5f17d60622572267cdf7c71fc563ac8e9c57bb934df01eb04f4648c1528b070a Jul 27 14:17:37.759175 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.759009 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw" Jul 27 14:17:37.796451 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.796376 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s"] Jul 27 14:17:37.800226 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:17:37.800192 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf86b19c8_ef8a_4e72_8048_c9fe8112d3ec.slice/crio-fe0014c556dc038df3ea20042ac34873ab7c534a6cb8b03e8862d8a9af8145cd WatchSource:0}: Error finding container fe0014c556dc038df3ea20042ac34873ab7c534a6cb8b03e8862d8a9af8145cd: Status 404 returned error can't find the container with id fe0014c556dc038df3ea20042ac34873ab7c534a6cb8b03e8862d8a9af8145cd Jul 27 14:17:37.823803 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.823741 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-nhx74"] Jul 27 14:17:37.829408 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:17:37.829368 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5170c1_44f1_4ee2_a373_78788f49e622.slice/crio-43aa21577071c306510fc09175a634ec5eca11b0dfc7aacb90ab8867f8d09ec2 WatchSource:0}: Error finding container 43aa21577071c306510fc09175a634ec5eca11b0dfc7aacb90ab8867f8d09ec2: Status 404 returned error can't find the container with id 43aa21577071c306510fc09175a634ec5eca11b0dfc7aacb90ab8867f8d09ec2 Jul 27 14:17:37.871179 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.871150 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd"] Jul 27 14:17:37.874039 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:17:37.874004 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7678c3a4_0414_4678_ade4_767be3864a91.slice/crio-16a7cd1496217879ec3e57aa34d14b8afaec47f19c2de4c31ad3aae37062ee32 WatchSource:0}: Error finding container 16a7cd1496217879ec3e57aa34d14b8afaec47f19c2de4c31ad3aae37062ee32: Status 404 returned error can't find the container with id 16a7cd1496217879ec3e57aa34d14b8afaec47f19c2de4c31ad3aae37062ee32 Jul 27 14:17:37.897881 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.897859 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wdpj7"] Jul 27 14:17:37.899949 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:17:37.899928 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76fb3e7b_bef0_473d_a98e_0cbac98c891c.slice/crio-c854bc773571a43d9d5caf44acab80c3d594971cdd73fc23c93135067c47edcc WatchSource:0}: Error finding container c854bc773571a43d9d5caf44acab80c3d594971cdd73fc23c93135067c47edcc: Status 404 returned error can't find the container with id c854bc773571a43d9d5caf44acab80c3d594971cdd73fc23c93135067c47edcc Jul 27 14:17:37.939264 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:37.939217 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw"] Jul 27 14:17:37.941219 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:17:37.941196 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723b1610_1571_4007_8a24_6451d3bf2153.slice/crio-1b9403d8b3ad9d910908292fb5d8682fa76177bea934ac3e21d20792a1fc57b6 WatchSource:0}: Error finding container 1b9403d8b3ad9d910908292fb5d8682fa76177bea934ac3e21d20792a1fc57b6: Status 404 returned error can't find the container with id 1b9403d8b3ad9d910908292fb5d8682fa76177bea934ac3e21d20792a1fc57b6 Jul 27 14:17:38.005553 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.005467 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:38.005553 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.005514 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:38.005733 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.005613 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:38.005733 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.005615 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 14:17:38.005733 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.005704 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls podName:878dade8-dbdf-4710-9b30-b3470dcb4221 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:39.005686736 +0000 UTC m=+42.372948762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls") pod "cluster-samples-operator-6748489656-nhl9j" (UID: "878dade8-dbdf-4710-9b30-b3470dcb4221") : secret "samples-operator-tls" not found Jul 27 14:17:38.005733 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.005719 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls podName:028ffee0-b008-47b4-93b0-996dfbd4f7b1 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:39.005713103 +0000 UTC m=+42.372975129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fwmw7" (UID: "028ffee0-b008-47b4-93b0-996dfbd4f7b1") : secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:38.106392 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.106362 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:38.106535 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.106460 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:38.106535 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.106524 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 14:17:38.106626 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.106575 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:39.106563063 +0000 UTC m=+42.473825089 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : secret "router-metrics-certs-default" not found Jul 27 14:17:38.106703 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.106654 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:39.106635973 +0000 UTC m=+42.473898002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : configmap references non-existent config key: service-ca.crt Jul 27 14:17:38.542789 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.542747 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" event={"ID":"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec","Type":"ContainerStarted","Data":"fe0014c556dc038df3ea20042ac34873ab7c534a6cb8b03e8862d8a9af8145cd"} Jul 27 14:17:38.547135 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.547102 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-87smn" event={"ID":"acf16aa1-2fc6-4275-9845-a26701408128","Type":"ContainerStarted","Data":"f607886a086e5bd49d732b0a533b33c5a53844dad30dd457defc7b93c68610f9"} Jul 27 14:17:38.549121 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.549090 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" event={"ID":"a9ba173c-70f2-4c02-a10d-f48753f7f720","Type":"ContainerStarted","Data":"5f17d60622572267cdf7c71fc563ac8e9c57bb934df01eb04f4648c1528b070a"} Jul 27 14:17:38.550487 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.550428 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw" event={"ID":"723b1610-1571-4007-8a24-6451d3bf2153","Type":"ContainerStarted","Data":"1b9403d8b3ad9d910908292fb5d8682fa76177bea934ac3e21d20792a1fc57b6"} Jul 27 14:17:38.551831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.551806 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wdpj7" event={"ID":"76fb3e7b-bef0-473d-a98e-0cbac98c891c","Type":"ContainerStarted","Data":"c854bc773571a43d9d5caf44acab80c3d594971cdd73fc23c93135067c47edcc"} Jul 27 14:17:38.553594 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.553566 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd" event={"ID":"7678c3a4-0414-4678-ade4-767be3864a91","Type":"ContainerStarted","Data":"16a7cd1496217879ec3e57aa34d14b8afaec47f19c2de4c31ad3aae37062ee32"} Jul 27 14:17:38.554905 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.554880 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" event={"ID":"db5170c1-44f1-4ee2-a373-78788f49e622","Type":"ContainerStarted","Data":"43aa21577071c306510fc09175a634ec5eca11b0dfc7aacb90ab8867f8d09ec2"} Jul 27 14:17:38.579475 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.579407 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-87smn" podStartSLOduration=3.509511067 podStartE2EDuration="40.579391477s" totalImagesPullingTime="37.06988041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-27 14:17:38.575713157 +0000 UTC m=+41.942975222" watchObservedRunningTime="2026-07-27 14:17:38.579391477 +0000 UTC m=+41.946653524" Jul 27 14:17:38.814899 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.814225 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:38.814899 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.814372 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 14:17:38.814899 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.814388 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f6585d46-wlhhh: secret "image-registry-tls" not found Jul 27 14:17:38.814899 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.814440 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls podName:3516fbc8-37ce-4433-bbff-4faea6399efe nodeName:}" failed. No retries permitted until 2026-07-27 14:17:46.814425301 +0000 UTC m=+50.181687339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls") pod "image-registry-76f6585d46-wlhhh" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe") : secret "image-registry-tls" not found Jul 27 14:17:38.914756 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:38.914711 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:38.915468 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.915036 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 14:17:38.915468 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:38.915118 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls podName:00a47fc4-008e-4b66-8ae2-c5fed963599a nodeName:}" failed. No retries permitted until 2026-07-27 14:17:46.915097338 +0000 UTC m=+50.282359365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls") pod "dns-default-8cbvq" (UID: "00a47fc4-008e-4b66-8ae2-c5fed963599a") : secret "dns-default-metrics-tls" not found Jul 27 14:17:39.016252 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:39.016208 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:39.016437 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:39.016271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:39.016568 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:39.016550 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:39.016694 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:39.016620 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls podName:028ffee0-b008-47b4-93b0-996dfbd4f7b1 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:41.016601545 +0000 UTC m=+44.383863574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fwmw7" (UID: "028ffee0-b008-47b4-93b0-996dfbd4f7b1") : secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:39.017010 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:39.016988 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 14:17:39.017077 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:39.017044 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls podName:878dade8-dbdf-4710-9b30-b3470dcb4221 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:41.017029576 +0000 UTC m=+44.384291617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls") pod "cluster-samples-operator-6748489656-nhl9j" (UID: "878dade8-dbdf-4710-9b30-b3470dcb4221") : secret "samples-operator-tls" not found Jul 27 14:17:39.117134 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:39.117054 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:39.117320 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:39.117147 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:39.117320 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:39.117259 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 14:17:39.117320 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:39.117306 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:41.117293313 +0000 UTC m=+44.484555340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : secret "router-metrics-certs-default" not found Jul 27 14:17:39.117727 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:39.117689 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:41.117672242 +0000 UTC m=+44.484934294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : configmap references non-existent config key: service-ca.crt Jul 27 14:17:41.037772 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:41.037735 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:41.038176 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:41.037792 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:41.038176 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:41.037919 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 14:17:41.038176 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:41.037963 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:41.038176 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:41.037987 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls podName:878dade8-dbdf-4710-9b30-b3470dcb4221 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:45.037971781 +0000 UTC m=+48.405233807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls") pod "cluster-samples-operator-6748489656-nhl9j" (UID: "878dade8-dbdf-4710-9b30-b3470dcb4221") : secret "samples-operator-tls" not found Jul 27 14:17:41.038176 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:41.038012 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls podName:028ffee0-b008-47b4-93b0-996dfbd4f7b1 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:45.037999993 +0000 UTC m=+48.405262043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fwmw7" (UID: "028ffee0-b008-47b4-93b0-996dfbd4f7b1") : secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:41.138536 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:41.138496 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:41.138720 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:41.138623 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:41.138720 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:41.138686 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:45.138651574 +0000 UTC m=+48.505913616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : configmap references non-existent config key: service-ca.crt Jul 27 14:17:41.138852 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:41.138743 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 14:17:41.138852 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:41.138778 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:45.138766963 +0000 UTC m=+48.506028988 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : secret "router-metrics-certs-default" not found Jul 27 14:17:43.572722 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.572688 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" event={"ID":"db5170c1-44f1-4ee2-a373-78788f49e622","Type":"ContainerStarted","Data":"c355858f9fa7a3905f5b49a4b789787ed6a07f1ddedaa20019a423ce95f4e3c8"} Jul 27 14:17:43.573161 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.572980 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:43.574436 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.574407 2555 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-nhx74 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" start-of-body= Jul 27 14:17:43.574550 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.574501 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" event={"ID":"a9ba173c-70f2-4c02-a10d-f48753f7f720","Type":"ContainerStarted","Data":"14b53bd611d3e3991c26c8bbfb6a6b3d7bce3467d501142f830ec089ee894ed7"} Jul 27 14:17:43.574550 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.574506 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" podUID="db5170c1-44f1-4ee2-a373-78788f49e622" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" Jul 27 14:17:43.576350 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.576329 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wdpj7" event={"ID":"76fb3e7b-bef0-473d-a98e-0cbac98c891c","Type":"ContainerStarted","Data":"56cde6dcdf21abeed69d59d2ff152ae81b15df647b0a407d5adc1e9b23a29a71"} Jul 27 14:17:43.577918 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.577896 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd" event={"ID":"7678c3a4-0414-4678-ade4-767be3864a91","Type":"ContainerStarted","Data":"c8476942b28b85a6590223699a7a929ee5c32624279cbc642365434bdb429eff"} Jul 27 14:17:43.592384 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.592331 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" podStartSLOduration=1.134146755 podStartE2EDuration="6.592313361s" totalImagesPullingTime="5.458166606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:43.591607112 +0000 UTC m=+46.958869162" watchObservedRunningTime="2026-07-27 14:17:43.592313361 +0000 UTC m=+46.959575410" Jul 27 14:17:43.610009 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.609953 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" podStartSLOduration=1.064431814 podStartE2EDuration="6.609932027s" totalImagesPullingTime="5.545500213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:43.609915598 +0000 UTC m=+46.977177647" watchObservedRunningTime="2026-07-27 14:17:43.609932027 +0000 UTC m=+46.977194281" Jul 27 14:17:43.644346 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.644177 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-wdpj7" podStartSLOduration=1.256021463 podStartE2EDuration="6.644158345s" totalImagesPullingTime="5.388136882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:43.64365264 +0000 UTC m=+47.010914689" watchObservedRunningTime="2026-07-27 14:17:43.644158345 +0000 UTC m=+47.011420393" Jul 27 14:17:43.644568 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:43.644536 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-z4nfd" podStartSLOduration=1.234074995 podStartE2EDuration="6.644528219s" totalImagesPullingTime="5.410453224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:43.626034267 +0000 UTC m=+46.993296318" watchObservedRunningTime="2026-07-27 14:17:43.644528219 +0000 UTC m=+47.011790266" Jul 27 14:17:44.583618 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:44.583580 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw" event={"ID":"723b1610-1571-4007-8a24-6451d3bf2153","Type":"ContainerStarted","Data":"722bec0a22a35468c767c246034dbcb3e48c764eebf08cf08bf7ebdf57209e48"} Jul 27 14:17:44.585194 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:44.585171 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/0.log" Jul 27 14:17:44.585332 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:44.585215 2555 generic.go:356] "Generic (PLEG): container finished" podID="db5170c1-44f1-4ee2-a373-78788f49e622" containerID="c355858f9fa7a3905f5b49a4b789787ed6a07f1ddedaa20019a423ce95f4e3c8" exitCode=255 Jul 27 14:17:44.585332 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:44.585268 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" event={"ID":"db5170c1-44f1-4ee2-a373-78788f49e622","Type":"ContainerDied","Data":"c355858f9fa7a3905f5b49a4b789787ed6a07f1ddedaa20019a423ce95f4e3c8"} Jul 27 14:17:44.585559 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:44.585541 2555 scope.go:117] "RemoveContainer" containerID="c355858f9fa7a3905f5b49a4b789787ed6a07f1ddedaa20019a423ce95f4e3c8" Jul 27 14:17:44.619052 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:44.618963 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4fqqw" podStartSLOduration=1.412067012 podStartE2EDuration="7.61891431s" totalImagesPullingTime="6.206847298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:44.618514087 +0000 UTC m=+47.985776132" watchObservedRunningTime="2026-07-27 14:17:44.61891431 +0000 UTC m=+47.986176361" Jul 27 14:17:45.054953 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.053641 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ssv66"] Jul 27 14:17:45.071845 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.071583 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ssv66"] Jul 27 14:17:45.072081 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.072057 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.076108 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.075041 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f5xtg\"" Jul 27 14:17:45.076108 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.075612 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 27 14:17:45.076108 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.075883 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 27 14:17:45.076339 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.076283 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.076425 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.076336 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7a7e56f1-e4f8-41da-afff-5b04280cb26e-crio-socket\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.076425 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.076405 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fkdb\" (UniqueName: \"kubernetes.io/projected/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-api-access-8fkdb\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.076524 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.076446 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a7e56f1-e4f8-41da-afff-5b04280cb26e-data-volume\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.076576 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.076561 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:45.076838 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.076594 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.076838 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.076633 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:45.076838 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.076787 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:45.077023 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.076861 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls podName:028ffee0-b008-47b4-93b0-996dfbd4f7b1 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:53.076842417 +0000 UTC m=+56.444104465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fwmw7" (UID: "028ffee0-b008-47b4-93b0-996dfbd4f7b1") : secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:45.078813 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.077233 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 14:17:45.078813 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.077306 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls podName:878dade8-dbdf-4710-9b30-b3470dcb4221 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:53.077282622 +0000 UTC m=+56.444544652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls") pod "cluster-samples-operator-6748489656-nhl9j" (UID: "878dade8-dbdf-4710-9b30-b3470dcb4221") : secret "samples-operator-tls" not found Jul 27 14:17:45.177893 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.177858 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fkdb\" (UniqueName: \"kubernetes.io/projected/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-api-access-8fkdb\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.178085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.177910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a7e56f1-e4f8-41da-afff-5b04280cb26e-data-volume\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.178085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.177955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:45.178085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.178041 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.178253 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.178091 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:45.178253 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.178123 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.178253 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.178159 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:53.178134117 +0000 UTC m=+56.545396160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : configmap references non-existent config key: service-ca.crt Jul 27 14:17:45.178253 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.178206 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 14:17:45.178253 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.178223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7a7e56f1-e4f8-41da-afff-5b04280cb26e-crio-socket\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.178253 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.178245 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a7e56f1-e4f8-41da-afff-5b04280cb26e-data-volume\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.178540 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.178286 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 14:17:45.178540 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.178306 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls podName:7a7e56f1-e4f8-41da-afff-5b04280cb26e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:45.678288977 +0000 UTC m=+49.045551014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssv66" (UID: "7a7e56f1-e4f8-41da-afff-5b04280cb26e") : secret "insights-runtime-extractor-tls" not found Jul 27 14:17:45.178540 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.178331 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm podName:7a7e56f1-e4f8-41da-afff-5b04280cb26e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:45.678317173 +0000 UTC m=+49.045579214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssv66" (UID: "7a7e56f1-e4f8-41da-afff-5b04280cb26e") : configmap "kube-rbac-proxy" not found Jul 27 14:17:45.178540 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.178405 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 14:17:45.178540 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.178439 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:17:53.178427657 +0000 UTC m=+56.545689698 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : secret "router-metrics-certs-default" not found Jul 27 14:17:45.178540 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.178486 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7a7e56f1-e4f8-41da-afff-5b04280cb26e-crio-socket\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.187074 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.187044 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fkdb\" (UniqueName: \"kubernetes.io/projected/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-api-access-8fkdb\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.591008 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.590982 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:17:45.591493 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.591421 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/0.log" Jul 27 14:17:45.591493 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.591457 2555 generic.go:356] "Generic (PLEG): container finished" podID="db5170c1-44f1-4ee2-a373-78788f49e622" containerID="2eb270fd299b0f23fd9cd37d687fcb0f0b6e70cef08b2f212d058d56940f4019" exitCode=255 Jul 27 14:17:45.591588 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.591523 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" event={"ID":"db5170c1-44f1-4ee2-a373-78788f49e622","Type":"ContainerDied","Data":"2eb270fd299b0f23fd9cd37d687fcb0f0b6e70cef08b2f212d058d56940f4019"} Jul 27 14:17:45.591588 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.591576 2555 scope.go:117] "RemoveContainer" containerID="c355858f9fa7a3905f5b49a4b789787ed6a07f1ddedaa20019a423ce95f4e3c8" Jul 27 14:17:45.591828 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.591804 2555 scope.go:117] "RemoveContainer" containerID="2eb270fd299b0f23fd9cd37d687fcb0f0b6e70cef08b2f212d058d56940f4019" Jul 27 14:17:45.592062 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.592036 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-nhx74_openshift-console-operator(db5170c1-44f1-4ee2-a373-78788f49e622)\"" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" podUID="db5170c1-44f1-4ee2-a373-78788f49e622" Jul 27 14:17:45.593316 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.593293 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" event={"ID":"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec","Type":"ContainerStarted","Data":"9afa26bc25c189ae1979339ba64adb715e6abcd9b2e442cc6c448c588d3a484d"} Jul 27 14:17:45.626231 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.626185 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" podStartSLOduration=1.869232639 podStartE2EDuration="8.62616607s" totalImagesPullingTime="6.756933431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:17:45.625892654 +0000 UTC m=+48.993154714" watchObservedRunningTime="2026-07-27 14:17:45.62616607 +0000 UTC m=+48.993428118" Jul 27 14:17:45.681700 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.681653 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.681883 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.681774 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 14:17:45.681948 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:45.681886 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:45.682201 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.682123 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 14:17:45.682201 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.682129 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm podName:7a7e56f1-e4f8-41da-afff-5b04280cb26e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:46.6821099 +0000 UTC m=+50.049371926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssv66" (UID: "7a7e56f1-e4f8-41da-afff-5b04280cb26e") : configmap "kube-rbac-proxy" not found Jul 27 14:17:45.682376 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:45.682211 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls podName:7a7e56f1-e4f8-41da-afff-5b04280cb26e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:46.682193355 +0000 UTC m=+50.049455386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssv66" (UID: "7a7e56f1-e4f8-41da-afff-5b04280cb26e") : secret "insights-runtime-extractor-tls" not found Jul 27 14:17:46.598085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:46.598058 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:17:46.598535 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:46.598381 2555 scope.go:117] "RemoveContainer" containerID="2eb270fd299b0f23fd9cd37d687fcb0f0b6e70cef08b2f212d058d56940f4019" Jul 27 14:17:46.598604 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.598539 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-nhx74_openshift-console-operator(db5170c1-44f1-4ee2-a373-78788f49e622)\"" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" podUID="db5170c1-44f1-4ee2-a373-78788f49e622" Jul 27 14:17:46.692976 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:46.692934 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:46.693183 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.693054 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 14:17:46.693183 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.693120 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm podName:7a7e56f1-e4f8-41da-afff-5b04280cb26e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:48.693102117 +0000 UTC m=+52.060364143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssv66" (UID: "7a7e56f1-e4f8-41da-afff-5b04280cb26e") : configmap "kube-rbac-proxy" not found Jul 27 14:17:46.693183 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:46.693164 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:46.693369 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.693335 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 14:17:46.693456 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.693438 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls podName:7a7e56f1-e4f8-41da-afff-5b04280cb26e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:48.693425003 +0000 UTC m=+52.060687030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssv66" (UID: "7a7e56f1-e4f8-41da-afff-5b04280cb26e") : secret "insights-runtime-extractor-tls" not found Jul 27 14:17:46.895458 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:46.895363 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:17:46.895603 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.895500 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 14:17:46.895603 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.895517 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f6585d46-wlhhh: secret "image-registry-tls" not found Jul 27 14:17:46.895603 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.895570 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls podName:3516fbc8-37ce-4433-bbff-4faea6399efe nodeName:}" failed. No retries permitted until 2026-07-27 14:18:02.895555845 +0000 UTC m=+66.262817904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls") pod "image-registry-76f6585d46-wlhhh" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe") : secret "image-registry-tls" not found Jul 27 14:17:46.996599 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:46.996562 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:17:46.996790 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.996739 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 14:17:46.996840 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:46.996818 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls podName:00a47fc4-008e-4b66-8ae2-c5fed963599a nodeName:}" failed. No retries permitted until 2026-07-27 14:18:02.996799689 +0000 UTC m=+66.364061735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls") pod "dns-default-8cbvq" (UID: "00a47fc4-008e-4b66-8ae2-c5fed963599a") : secret "dns-default-metrics-tls" not found Jul 27 14:17:47.242124 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:47.242094 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jbjwv_4bdecfbd-3e2f-47f0-a85b-2efffb7aef68/dns-node-resolver/0.log" Jul 27 14:17:47.630391 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:47.630289 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:47.630882 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:47.630848 2555 scope.go:117] "RemoveContainer" containerID="2eb270fd299b0f23fd9cd37d687fcb0f0b6e70cef08b2f212d058d56940f4019" Jul 27 14:17:47.631059 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:47.631034 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-nhx74_openshift-console-operator(db5170c1-44f1-4ee2-a373-78788f49e622)\"" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" podUID="db5170c1-44f1-4ee2-a373-78788f49e622" Jul 27 14:17:48.042569 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:48.042541 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gpfpw_def06289-55b1-475f-a338-42014932d174/node-ca/0.log" Jul 27 14:17:48.714034 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:48.713998 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:48.714421 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:48.714062 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:48.714421 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:48.714188 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 14:17:48.714421 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:48.714249 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls podName:7a7e56f1-e4f8-41da-afff-5b04280cb26e nodeName:}" failed. No retries permitted until 2026-07-27 14:17:52.714233115 +0000 UTC m=+56.081495157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssv66" (UID: "7a7e56f1-e4f8-41da-afff-5b04280cb26e") : secret "insights-runtime-extractor-tls" not found Jul 27 14:17:48.714525 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:48.714512 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a7e56f1-e4f8-41da-afff-5b04280cb26e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:48.842330 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:48.842301 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bwz8r_f0dfb067-bcb2-4d4c-88be-f93bec24022f/serve-healthcheck-canary/0.log" Jul 27 14:17:52.750682 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:52.750622 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:17:52.751274 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:52.750777 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 14:17:52.751274 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:52.750846 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls podName:7a7e56f1-e4f8-41da-afff-5b04280cb26e nodeName:}" failed. No retries permitted until 2026-07-27 14:18:00.75082707 +0000 UTC m=+64.118089096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssv66" (UID: "7a7e56f1-e4f8-41da-afff-5b04280cb26e") : secret "insights-runtime-extractor-tls" not found Jul 27 14:17:53.154905 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:53.154823 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:17:53.155050 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:53.154985 2555 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:53.155050 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:53.155003 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:17:53.155137 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:53.155052 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls podName:028ffee0-b008-47b4-93b0-996dfbd4f7b1 nodeName:}" failed. No retries permitted until 2026-07-27 14:18:09.155032568 +0000 UTC m=+72.522294619 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fwmw7" (UID: "028ffee0-b008-47b4-93b0-996dfbd4f7b1") : secret "cluster-monitoring-operator-tls" not found Jul 27 14:17:53.155137 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:53.155096 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 14:17:53.155217 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:53.155138 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls podName:878dade8-dbdf-4710-9b30-b3470dcb4221 nodeName:}" failed. No retries permitted until 2026-07-27 14:18:09.155127141 +0000 UTC m=+72.522389173 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls") pod "cluster-samples-operator-6748489656-nhl9j" (UID: "878dade8-dbdf-4710-9b30-b3470dcb4221") : secret "samples-operator-tls" not found Jul 27 14:17:53.255730 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:53.255677 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:53.255940 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:53.255862 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:17:53.255940 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:53.255887 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:18:09.255864608 +0000 UTC m=+72.623126639 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : configmap references non-existent config key: service-ca.crt Jul 27 14:17:53.255940 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:53.255932 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 14:17:53.256080 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:53.255977 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs podName:c5d14ecc-98e1-45cf-9071-0106bec58113 nodeName:}" failed. No retries permitted until 2026-07-27 14:18:09.255962589 +0000 UTC m=+72.623224614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs") pod "router-default-7745b5dc69-b8hmd" (UID: "c5d14ecc-98e1-45cf-9071-0106bec58113") : secret "router-metrics-certs-default" not found Jul 27 14:17:53.573807 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:53.573752 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:17:53.574167 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:53.574152 2555 scope.go:117] "RemoveContainer" containerID="2eb270fd299b0f23fd9cd37d687fcb0f0b6e70cef08b2f212d058d56940f4019" Jul 27 14:17:53.574307 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:17:53.574290 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-nhx74_openshift-console-operator(db5170c1-44f1-4ee2-a373-78788f49e622)\"" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" podUID="db5170c1-44f1-4ee2-a373-78788f49e622" Jul 27 14:17:57.514397 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:17:57.514366 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzr9g" Jul 27 14:18:00.823368 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:00.823324 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:18:00.825646 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:00.825627 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a7e56f1-e4f8-41da-afff-5b04280cb26e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssv66\" (UID: \"7a7e56f1-e4f8-41da-afff-5b04280cb26e\") " pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:18:00.991181 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:00.991148 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f5xtg\"" Jul 27 14:18:00.999349 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:00.999323 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ssv66" Jul 27 14:18:01.125271 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:01.125249 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ssv66"] Jul 27 14:18:01.127640 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:01.127617 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a7e56f1_e4f8_41da_afff_5b04280cb26e.slice/crio-d0f78d28e111d4616933d5ca33e9c82e4b2854a8c9af5e0a1aaa2ad6e28edeb5 WatchSource:0}: Error finding container d0f78d28e111d4616933d5ca33e9c82e4b2854a8c9af5e0a1aaa2ad6e28edeb5: Status 404 returned error can't find the container with id d0f78d28e111d4616933d5ca33e9c82e4b2854a8c9af5e0a1aaa2ad6e28edeb5 Jul 27 14:18:01.646722 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:01.646678 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssv66" event={"ID":"7a7e56f1-e4f8-41da-afff-5b04280cb26e","Type":"ContainerStarted","Data":"d0f78d28e111d4616933d5ca33e9c82e4b2854a8c9af5e0a1aaa2ad6e28edeb5"} Jul 27 14:18:01.933886 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:01.933178 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:18:01.933886 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:01.933325 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:18:01.936433 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:01.936230 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 14:18:01.936433 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:01.936349 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 14:18:01.947049 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:01.947019 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b27844f-7fba-4449-b514-1f2d9424f97e-original-pull-secret\") pod \"global-pull-secret-syncer-np467\" (UID: \"9b27844f-7fba-4449-b514-1f2d9424f97e\") " pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:18:01.948099 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:01.948065 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de-metrics-certs\") pod \"network-metrics-daemon-lxpdl\" (UID: \"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de\") " pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:18:02.033953 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.033910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:18:02.038737 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.038706 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mj97\" (UniqueName: \"kubernetes.io/projected/0b1c2c00-b265-42b8-b25c-28a4acd4f3c7-kube-api-access-2mj97\") pod \"network-check-target-tl44d\" (UID: \"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7\") " pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:18:02.157737 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.157653 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-np467" Jul 27 14:18:02.167843 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.167796 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mmpms\"" Jul 27 14:18:02.175471 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.175439 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:18:02.176286 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.176072 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nn5ms\"" Jul 27 14:18:02.184012 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.183738 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lxpdl" Jul 27 14:18:02.336971 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.336872 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-np467"] Jul 27 14:18:02.341172 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:02.341141 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b27844f_7fba_4449_b514_1f2d9424f97e.slice/crio-acbac2a83deef9553afa4e17b7d20c2d40354796cfbe82c6605e2193c4dffce6 WatchSource:0}: Error finding container acbac2a83deef9553afa4e17b7d20c2d40354796cfbe82c6605e2193c4dffce6: Status 404 returned error can't find the container with id acbac2a83deef9553afa4e17b7d20c2d40354796cfbe82c6605e2193c4dffce6 Jul 27 14:18:02.362736 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.362696 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tl44d"] Jul 27 14:18:02.367959 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:02.367919 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b1c2c00_b265_42b8_b25c_28a4acd4f3c7.slice/crio-c42175f3bb751f82ef2fe65e8d3cad840bb0244ea92887bc067c432b8639b990 WatchSource:0}: Error finding container c42175f3bb751f82ef2fe65e8d3cad840bb0244ea92887bc067c432b8639b990: Status 404 returned error can't find the container with id c42175f3bb751f82ef2fe65e8d3cad840bb0244ea92887bc067c432b8639b990 Jul 27 14:18:02.396800 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.396717 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lxpdl"] Jul 27 14:18:02.404096 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:02.404064 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f4ef8c_fc0a_4e3a_8e64_7c54a3dd19de.slice/crio-cc45dd7633f428989df18e9e06bc4af0e55fd6a862a4031397d1926eb7070a9d WatchSource:0}: Error finding container cc45dd7633f428989df18e9e06bc4af0e55fd6a862a4031397d1926eb7070a9d: Status 404 returned error can't find the container with id cc45dd7633f428989df18e9e06bc4af0e55fd6a862a4031397d1926eb7070a9d Jul 27 14:18:02.652230 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.652137 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-np467" event={"ID":"9b27844f-7fba-4449-b514-1f2d9424f97e","Type":"ContainerStarted","Data":"acbac2a83deef9553afa4e17b7d20c2d40354796cfbe82c6605e2193c4dffce6"} Jul 27 14:18:02.653590 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.653560 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lxpdl" event={"ID":"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de","Type":"ContainerStarted","Data":"cc45dd7633f428989df18e9e06bc4af0e55fd6a862a4031397d1926eb7070a9d"} Jul 27 14:18:02.655057 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.655028 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tl44d" event={"ID":"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7","Type":"ContainerStarted","Data":"9eadd44f8c73f7817c736810552f21e24b9982b95fa0110bea1404edbfd755b5"} Jul 27 14:18:02.655162 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.655077 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tl44d" event={"ID":"0b1c2c00-b265-42b8-b25c-28a4acd4f3c7","Type":"ContainerStarted","Data":"c42175f3bb751f82ef2fe65e8d3cad840bb0244ea92887bc067c432b8639b990"} Jul 27 14:18:02.655218 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.655180 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:18:02.943170 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.943139 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:18:02.945982 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:02.945962 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"image-registry-76f6585d46-wlhhh\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:18:03.044397 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.044363 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:18:03.047276 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.047243 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/00a47fc4-008e-4b66-8ae2-c5fed963599a-metrics-tls\") pod \"dns-default-8cbvq\" (UID: \"00a47fc4-008e-4b66-8ae2-c5fed963599a\") " pod="openshift-dns/dns-default-8cbvq" Jul 27 14:18:03.178859 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.178827 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qrv8d\"" Jul 27 14:18:03.186360 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.186329 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:18:03.208451 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.205034 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9kwzw\"" Jul 27 14:18:03.212996 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.212491 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8cbvq" Jul 27 14:18:03.370626 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.369406 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tl44d" podStartSLOduration=66.369385868 podStartE2EDuration="1m6.369385868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:02.671513888 +0000 UTC m=+66.038775937" watchObservedRunningTime="2026-07-27 14:18:03.369385868 +0000 UTC m=+66.736647897" Jul 27 14:18:03.370995 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.370968 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f6585d46-wlhhh"] Jul 27 14:18:03.374173 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:03.374126 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3516fbc8_37ce_4433_bbff_4faea6399efe.slice/crio-616e3526e5b9751700acd724115bf162bcbab0dee22ba7c61dfa8d42c400e09b WatchSource:0}: Error finding container 616e3526e5b9751700acd724115bf162bcbab0dee22ba7c61dfa8d42c400e09b: Status 404 returned error can't find the container with id 616e3526e5b9751700acd724115bf162bcbab0dee22ba7c61dfa8d42c400e09b Jul 27 14:18:03.400886 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.400640 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8cbvq"] Jul 27 14:18:03.407915 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:03.407883 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a47fc4_008e_4b66_8ae2_c5fed963599a.slice/crio-fba579164c2b265ddc36bed1dab9832957eee3b68bd75854c31d0b365bf64927 WatchSource:0}: Error finding container fba579164c2b265ddc36bed1dab9832957eee3b68bd75854c31d0b365bf64927: Status 404 returned error can't find the container with id fba579164c2b265ddc36bed1dab9832957eee3b68bd75854c31d0b365bf64927 Jul 27 14:18:03.662696 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.662115 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" event={"ID":"3516fbc8-37ce-4433-bbff-4faea6399efe","Type":"ContainerStarted","Data":"0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254"} Jul 27 14:18:03.662696 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.662175 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" event={"ID":"3516fbc8-37ce-4433-bbff-4faea6399efe","Type":"ContainerStarted","Data":"616e3526e5b9751700acd724115bf162bcbab0dee22ba7c61dfa8d42c400e09b"} Jul 27 14:18:03.662696 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.662353 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:18:03.663944 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.663845 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cbvq" event={"ID":"00a47fc4-008e-4b66-8ae2-c5fed963599a","Type":"ContainerStarted","Data":"fba579164c2b265ddc36bed1dab9832957eee3b68bd75854c31d0b365bf64927"} Jul 27 14:18:03.691821 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:03.691744 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" podStartSLOduration=65.691720411 podStartE2EDuration="1m5.691720411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:03.690902019 +0000 UTC m=+67.058164068" watchObservedRunningTime="2026-07-27 14:18:03.691720411 +0000 UTC m=+67.058982462" Jul 27 14:18:08.143119 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:08.143086 2555 scope.go:117] "RemoveContainer" containerID="2eb270fd299b0f23fd9cd37d687fcb0f0b6e70cef08b2f212d058d56940f4019" Jul 27 14:18:08.686483 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:08.686460 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:18:08.686597 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:08.686540 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" event={"ID":"db5170c1-44f1-4ee2-a373-78788f49e622","Type":"ContainerStarted","Data":"82194602f4091c6a760c25eea195588d31077e126d700aa6307c96b0ae74a1c2"} Jul 27 14:18:09.201856 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.201808 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:18:09.202348 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.201872 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:18:09.204783 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.204751 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/878dade8-dbdf-4710-9b30-b3470dcb4221-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nhl9j\" (UID: \"878dade8-dbdf-4710-9b30-b3470dcb4221\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:18:09.204934 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.204757 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/028ffee0-b008-47b4-93b0-996dfbd4f7b1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fwmw7\" (UID: \"028ffee0-b008-47b4-93b0-996dfbd4f7b1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:18:09.302909 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.302857 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:09.303089 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.302955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:09.303694 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.303642 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5d14ecc-98e1-45cf-9071-0106bec58113-service-ca-bundle\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:09.305976 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.305951 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d14ecc-98e1-45cf-9071-0106bec58113-metrics-certs\") pod \"router-default-7745b5dc69-b8hmd\" (UID: \"c5d14ecc-98e1-45cf-9071-0106bec58113\") " pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:09.354180 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.354143 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vsv7l\"" Jul 27 14:18:09.362405 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.362370 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" Jul 27 14:18:09.406891 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.406856 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ccjxz\"" Jul 27 14:18:09.415239 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.414973 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" Jul 27 14:18:09.500140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.500114 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vjx9h\"" Jul 27 14:18:09.508211 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.508182 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:09.680968 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.680943 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7"] Jul 27 14:18:09.700008 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.699881 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j"] Jul 27 14:18:09.702776 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.702701 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lxpdl" event={"ID":"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de","Type":"ContainerStarted","Data":"808eb43bd140049e9bf15a709dfd4ea9b1be4f74cdc44737db509ab21c36f4b3"} Jul 27 14:18:09.705839 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.705773 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-np467" event={"ID":"9b27844f-7fba-4449-b514-1f2d9424f97e","Type":"ContainerStarted","Data":"01d8768e46e9bf3371db53f106fcb653d11652160b2cc845545c405003132cf0"} Jul 27 14:18:09.709070 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.709040 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" event={"ID":"028ffee0-b008-47b4-93b0-996dfbd4f7b1","Type":"ContainerStarted","Data":"c7ef9d76da4703f6f6601dd24e453c2d9a450a9e4ac23dcefdb9487dd6545604"} Jul 27 14:18:09.709271 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.709243 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:18:09.714790 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.714763 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhx74" Jul 27 14:18:09.731090 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.731022 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-np467" podStartSLOduration=65.457922933 podStartE2EDuration="1m11.730986674s" totalImagesPullingTime="6.273063741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:09.730747498 +0000 UTC m=+73.098009547" watchObservedRunningTime="2026-07-27 14:18:09.730986674 +0000 UTC m=+73.098248723" Jul 27 14:18:09.734409 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:09.734382 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7745b5dc69-b8hmd"] Jul 27 14:18:09.896945 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:09.896914 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d14ecc_98e1_45cf_9071_0106bec58113.slice/crio-e23d1b0d7ba9b01eba032cd60f8c35ac7b703fa9b7e7dced27796bcdde8c7d9e WatchSource:0}: Error finding container e23d1b0d7ba9b01eba032cd60f8c35ac7b703fa9b7e7dced27796bcdde8c7d9e: Status 404 returned error can't find the container with id e23d1b0d7ba9b01eba032cd60f8c35ac7b703fa9b7e7dced27796bcdde8c7d9e Jul 27 14:18:10.717206 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.717173 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cbvq" event={"ID":"00a47fc4-008e-4b66-8ae2-c5fed963599a","Type":"ContainerStarted","Data":"c1e394a5d90a5bc14399963c7afc2246b541a9676398188c0a9fdfba43492203"} Jul 27 14:18:10.717710 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.717226 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cbvq" event={"ID":"00a47fc4-008e-4b66-8ae2-c5fed963599a","Type":"ContainerStarted","Data":"c03323220b59f5ea8544c609ae622f855614675b20a61c62132e5f519c6800cc"} Jul 27 14:18:10.717710 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.717292 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8cbvq" Jul 27 14:18:10.718710 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.718631 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" event={"ID":"878dade8-dbdf-4710-9b30-b3470dcb4221","Type":"ContainerStarted","Data":"5e3502a64930a9ecceb3751c9498c335e0c82eb7d60b6ce1224be5ec091d3876"} Jul 27 14:18:10.720404 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.720373 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssv66" event={"ID":"7a7e56f1-e4f8-41da-afff-5b04280cb26e","Type":"ContainerStarted","Data":"c2aeac58a5602e11615c943fbc9565cae9a01016cd5b448c799b4007e958cc8e"} Jul 27 14:18:10.722335 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.722309 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lxpdl" event={"ID":"a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de","Type":"ContainerStarted","Data":"d12fb7fbdfd3dc2ac7e4204a8445ca11b2d7bf3680bd7047972dbb08ac044151"} Jul 27 14:18:10.724247 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.724221 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" event={"ID":"c5d14ecc-98e1-45cf-9071-0106bec58113","Type":"ContainerStarted","Data":"8cd9869947560f6048ddac28debc02e3c99dcf95f4c1ac966e3f0742bedf084d"} Jul 27 14:18:10.724369 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.724269 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" event={"ID":"c5d14ecc-98e1-45cf-9071-0106bec58113","Type":"ContainerStarted","Data":"e23d1b0d7ba9b01eba032cd60f8c35ac7b703fa9b7e7dced27796bcdde8c7d9e"} Jul 27 14:18:10.738459 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.738398 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8cbvq" podStartSLOduration=33.224211741 podStartE2EDuration="39.738377254s" totalImagesPullingTime="6.514165513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:10.737158267 +0000 UTC m=+74.104420316" watchObservedRunningTime="2026-07-27 14:18:10.738377254 +0000 UTC m=+74.105639306" Jul 27 14:18:10.760092 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.760032 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" podStartSLOduration=33.76001024 podStartE2EDuration="33.76001024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:10.759202935 +0000 UTC m=+74.126464985" watchObservedRunningTime="2026-07-27 14:18:10.76001024 +0000 UTC m=+74.127272284" Jul 27 14:18:10.779390 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:10.779335 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lxpdl" podStartSLOduration=66.580792237 podStartE2EDuration="1m12.779318667s" totalImagesPullingTime="6.19852643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:10.778375983 +0000 UTC m=+74.145638032" watchObservedRunningTime="2026-07-27 14:18:10.779318667 +0000 UTC m=+74.146580926" Jul 27 14:18:11.508697 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:11.508647 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:11.511648 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:11.511606 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:11.728677 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:11.728620 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:11.730117 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:11.730093 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7745b5dc69-b8hmd" Jul 27 14:18:15.745454 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:15.745416 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" event={"ID":"878dade8-dbdf-4710-9b30-b3470dcb4221","Type":"ContainerStarted","Data":"2d548f136e87bbe19cc6040b828b9bcf3f4274bd83cca49603c90ee6179fb31c"} Jul 27 14:18:15.745967 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:15.745483 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" event={"ID":"878dade8-dbdf-4710-9b30-b3470dcb4221","Type":"ContainerStarted","Data":"5dd0bedcc9d5275de8cf640774e9ba098a5c88bc9f53c208956b0aad0e3bd0a3"} Jul 27 14:18:15.747312 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:15.747283 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssv66" event={"ID":"7a7e56f1-e4f8-41da-afff-5b04280cb26e","Type":"ContainerStarted","Data":"a0eec5f03714c41374872f80914982689da91fd53aaa0644e09d1e6ba1610b3b"} Jul 27 14:18:15.748836 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:15.748814 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" event={"ID":"028ffee0-b008-47b4-93b0-996dfbd4f7b1","Type":"ContainerStarted","Data":"e232ff78de251f39dd525f60673892f4a4f0f9dc08033230ca81cc683fc9458a"} Jul 27 14:18:15.771852 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:15.771792 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nhl9j" podStartSLOduration=33.382110572 podStartE2EDuration="38.771770052s" totalImagesPullingTime="5.38965948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:15.770917616 +0000 UTC m=+79.138179658" watchObservedRunningTime="2026-07-27 14:18:15.771770052 +0000 UTC m=+79.139032093" Jul 27 14:18:15.818099 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:15.817994 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fwmw7" podStartSLOduration=33.218930804 podStartE2EDuration="38.817978742s" totalImagesPullingTime="5.599047938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:15.817086125 +0000 UTC m=+79.184348172" watchObservedRunningTime="2026-07-27 14:18:15.817978742 +0000 UTC m=+79.185240790" Jul 27 14:18:19.766047 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:19.766005 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssv66" event={"ID":"7a7e56f1-e4f8-41da-afff-5b04280cb26e","Type":"ContainerStarted","Data":"7f8a15f605b003c7bdca0a461453981f4b817f038d4229e7a3aed7e580da9742"} Jul 27 14:18:19.794757 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:19.794694 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ssv66" podStartSLOduration=17.0404292 podStartE2EDuration="34.794673121s" totalImagesPullingTime="17.754243921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:17:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:19.794548529 +0000 UTC m=+83.161810577" watchObservedRunningTime="2026-07-27 14:18:19.794673121 +0000 UTC m=+83.161935168" Jul 27 14:18:20.730689 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:20.730636 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8cbvq" Jul 27 14:18:23.191343 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:23.191305 2555 patch_prober.go:28] interesting pod/image-registry-76f6585d46-wlhhh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 27 14:18:23.191799 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:23.191357 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" podUID="3516fbc8-37ce-4433-bbff-4faea6399efe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 14:18:24.672399 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:24.672364 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:18:32.076398 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:32.076354 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76f6585d46-wlhhh"] Jul 27 14:18:33.666741 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:33.666704 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tl44d" Jul 27 14:18:37.378122 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.378086 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5"] Jul 27 14:18:37.382806 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.382784 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.386545 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.386501 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kjv69\"" Jul 27 14:18:37.386545 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.386500 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 27 14:18:37.387888 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.387870 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 27 14:18:37.387996 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.387888 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 27 14:18:37.387996 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.387928 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 27 14:18:37.396004 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.395984 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-txldj"] Jul 27 14:18:37.399108 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.399089 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.404074 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.404054 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5jzs5\"" Jul 27 14:18:37.404174 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.404102 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 27 14:18:37.404174 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.404133 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 27 14:18:37.404285 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.404184 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 27 14:18:37.413160 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.413140 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5"] Jul 27 14:18:37.527232 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527193 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.527232 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527238 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.527491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527264 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j74w\" (UniqueName: \"kubernetes.io/projected/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-api-access-2j74w\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.527491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527286 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-wtmp\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527302 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw2sh\" (UniqueName: \"kubernetes.io/projected/0f366d0f-4f55-4753-be5e-9e781463cc22-kube-api-access-cw2sh\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527318 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-tls\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527340 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.527491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527361 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-sys\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527381 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-accelerators-collector-config\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527468 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f366d0f-4f55-4753-be5e-9e781463cc22-metrics-client-ca\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527855 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527505 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-root\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527855 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527542 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-textfile\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527855 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527567 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.527855 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527600 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.527855 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.527631 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.628479 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw2sh\" (UniqueName: \"kubernetes.io/projected/0f366d0f-4f55-4753-be5e-9e781463cc22-kube-api-access-cw2sh\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628479 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628430 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-tls\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628479 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628453 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.628797 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628484 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-sys\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628797 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628504 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-accelerators-collector-config\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628797 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628530 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f366d0f-4f55-4753-be5e-9e781463cc22-metrics-client-ca\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628797 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628546 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-root\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628797 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628572 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-textfile\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628797 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628596 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628797 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628651 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-sys\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.628797 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628677 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-root\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.629186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628690 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.629186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628951 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-textfile\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.629186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628959 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.629186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.628993 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.629186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629035 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.629186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629081 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j74w\" (UniqueName: \"kubernetes.io/projected/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-api-access-2j74w\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.629186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629109 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-wtmp\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.629534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629264 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-wtmp\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.629534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629267 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-accelerators-collector-config\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.629534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629312 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.629763 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629735 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.629985 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629963 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f366d0f-4f55-4753-be5e-9e781463cc22-metrics-client-ca\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.630070 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.629969 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.631329 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.631299 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-tls\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.631431 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.631391 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f366d0f-4f55-4753-be5e-9e781463cc22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.631535 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.631514 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.631801 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.631784 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.642291 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.642267 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw2sh\" (UniqueName: \"kubernetes.io/projected/0f366d0f-4f55-4753-be5e-9e781463cc22-kube-api-access-cw2sh\") pod \"node-exporter-txldj\" (UID: \"0f366d0f-4f55-4753-be5e-9e781463cc22\") " pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.642378 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.642358 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j74w\" (UniqueName: \"kubernetes.io/projected/793e6f62-0fe6-46ca-aae0-2a68e2000fd9-kube-api-access-2j74w\") pod \"kube-state-metrics-56d9f944b4-7ppt5\" (UID: \"793e6f62-0fe6-46ca-aae0-2a68e2000fd9\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.691935 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.691888 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" Jul 27 14:18:37.707416 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.707390 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-txldj" Jul 27 14:18:37.718684 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:37.718633 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f366d0f_4f55_4753_be5e_9e781463cc22.slice/crio-cf20524a539ac55e618b11230c076ac9f4e83ce3b54ae660a34fe2903c1b4e1d WatchSource:0}: Error finding container cf20524a539ac55e618b11230c076ac9f4e83ce3b54ae660a34fe2903c1b4e1d: Status 404 returned error can't find the container with id cf20524a539ac55e618b11230c076ac9f4e83ce3b54ae660a34fe2903c1b4e1d Jul 27 14:18:37.834626 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.834593 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-txldj" event={"ID":"0f366d0f-4f55-4753-be5e-9e781463cc22","Type":"ContainerStarted","Data":"cf20524a539ac55e618b11230c076ac9f4e83ce3b54ae660a34fe2903c1b4e1d"} Jul 27 14:18:37.835440 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:37.835419 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5"] Jul 27 14:18:37.837739 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:37.837713 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793e6f62_0fe6_46ca_aae0_2a68e2000fd9.slice/crio-d0792a5bf7a80d47d39e4b444a6e9246e6cff5c63e7da3e4ec04d78dc5dfaeae WatchSource:0}: Error finding container d0792a5bf7a80d47d39e4b444a6e9246e6cff5c63e7da3e4ec04d78dc5dfaeae: Status 404 returned error can't find the container with id d0792a5bf7a80d47d39e4b444a6e9246e6cff5c63e7da3e4ec04d78dc5dfaeae Jul 27 14:18:38.839783 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:38.839738 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" event={"ID":"793e6f62-0fe6-46ca-aae0-2a68e2000fd9","Type":"ContainerStarted","Data":"d0792a5bf7a80d47d39e4b444a6e9246e6cff5c63e7da3e4ec04d78dc5dfaeae"} Jul 27 14:18:40.349306 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.349197 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf"] Jul 27 14:18:40.367333 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.367297 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.372206 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.372156 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 27 14:18:40.372206 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.372197 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 27 14:18:40.372438 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.372206 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 27 14:18:40.373187 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.372954 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf"] Jul 27 14:18:40.373187 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.373077 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 27 14:18:40.373442 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.373422 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 27 14:18:40.373442 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.373438 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8fipjea0ld8sr\"" Jul 27 14:18:40.373568 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.373489 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6d4r7\"" Jul 27 14:18:40.449984 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.449901 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.449984 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.449940 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.450173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.450017 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-grpc-tls\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.450173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.450043 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcw47\" (UniqueName: \"kubernetes.io/projected/69f241b0-2404-441a-93f7-97857c9b2da8-kube-api-access-vcw47\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.450173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.450078 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-tls\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.450173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.450100 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f241b0-2404-441a-93f7-97857c9b2da8-metrics-client-ca\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.450173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.450138 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.450375 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.450173 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.550954 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.550926 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-grpc-tls\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.551080 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.550968 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcw47\" (UniqueName: \"kubernetes.io/projected/69f241b0-2404-441a-93f7-97857c9b2da8-kube-api-access-vcw47\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.551080 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.551004 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-tls\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.551080 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.551028 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f241b0-2404-441a-93f7-97857c9b2da8-metrics-client-ca\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.551234 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.551110 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.551234 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.551148 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.551234 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.551199 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.551383 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.551242 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.552051 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.551923 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f241b0-2404-441a-93f7-97857c9b2da8-metrics-client-ca\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.554063 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.554012 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-tls\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.555066 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.554995 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.555244 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.555216 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.556014 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.555969 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.556014 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.555994 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-grpc-tls\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.556302 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.556277 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69f241b0-2404-441a-93f7-97857c9b2da8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.561564 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.561537 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcw47\" (UniqueName: \"kubernetes.io/projected/69f241b0-2404-441a-93f7-97857c9b2da8-kube-api-access-vcw47\") pod \"thanos-querier-5dfbfbd66c-pfzwf\" (UID: \"69f241b0-2404-441a-93f7-97857c9b2da8\") " pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.698155 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.698035 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:40.848843 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.848805 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" event={"ID":"793e6f62-0fe6-46ca-aae0-2a68e2000fd9","Type":"ContainerStarted","Data":"f10ee9ea722152f44fe98a6ebe3ea75fbc0ed130653ab9cec7773b3b7f3b143d"} Jul 27 14:18:40.848990 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.848876 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" event={"ID":"793e6f62-0fe6-46ca-aae0-2a68e2000fd9","Type":"ContainerStarted","Data":"dce7ed79822d75ac285d3e9dcb5812922fc036c9e8bb76300255bb57477bdd53"} Jul 27 14:18:40.848990 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.848908 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" event={"ID":"793e6f62-0fe6-46ca-aae0-2a68e2000fd9","Type":"ContainerStarted","Data":"70148dfe0cd661f6c4b68a8d395b748222121ff336c742f17aed31e3fd5f954c"} Jul 27 14:18:40.870954 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.870895 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ppt5" podStartSLOduration=1.512161607 podStartE2EDuration="3.870872119s" totalImagesPullingTime="2.358710512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:40.869987152 +0000 UTC m=+104.237249202" watchObservedRunningTime="2026-07-27 14:18:40.870872119 +0000 UTC m=+104.238134170" Jul 27 14:18:40.954334 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:40.954305 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf"] Jul 27 14:18:40.956150 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:40.956125 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69f241b0_2404_441a_93f7_97857c9b2da8.slice/crio-81928f0757ea086e02170a8384fc41de73b2a4d7c52279ff4a9eeedd57871ac1 WatchSource:0}: Error finding container 81928f0757ea086e02170a8384fc41de73b2a4d7c52279ff4a9eeedd57871ac1: Status 404 returned error can't find the container with id 81928f0757ea086e02170a8384fc41de73b2a4d7c52279ff4a9eeedd57871ac1 Jul 27 14:18:41.685671 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.685634 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b5f6b554c-46flw"] Jul 27 14:18:41.726314 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.726276 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b5f6b554c-46flw"] Jul 27 14:18:41.726488 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.726428 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.729672 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.729633 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b3tldqmcq83t\"" Jul 27 14:18:41.729812 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.729633 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 27 14:18:41.730582 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.730565 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 27 14:18:41.730582 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.730575 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 27 14:18:41.730835 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.730820 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4dlkb\"" Jul 27 14:18:41.730953 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.730932 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 27 14:18:41.760045 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.760016 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/268d2adf-43d2-4ab0-ae70-f86855843ab9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.760045 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.760046 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-client-ca-bundle\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.760273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.760073 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-secret-metrics-server-client-certs\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.760273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.760093 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlc4f\" (UniqueName: \"kubernetes.io/projected/268d2adf-43d2-4ab0-ae70-f86855843ab9-kube-api-access-qlc4f\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.760273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.760167 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/268d2adf-43d2-4ab0-ae70-f86855843ab9-audit-log\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.760273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.760248 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/268d2adf-43d2-4ab0-ae70-f86855843ab9-metrics-server-audit-profiles\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.760427 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.760283 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-secret-metrics-server-tls\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.854679 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.854612 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" event={"ID":"69f241b0-2404-441a-93f7-97857c9b2da8","Type":"ContainerStarted","Data":"81928f0757ea086e02170a8384fc41de73b2a4d7c52279ff4a9eeedd57871ac1"} Jul 27 14:18:41.856208 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.856183 2555 generic.go:356] "Generic (PLEG): container finished" podID="0f366d0f-4f55-4753-be5e-9e781463cc22" containerID="772afc238c5abe75b28141099c9865aae5b3b2ad8b1e60c16f062adcbb47fe10" exitCode=0 Jul 27 14:18:41.856329 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.856268 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-txldj" event={"ID":"0f366d0f-4f55-4753-be5e-9e781463cc22","Type":"ContainerDied","Data":"772afc238c5abe75b28141099c9865aae5b3b2ad8b1e60c16f062adcbb47fe10"} Jul 27 14:18:41.860991 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.860961 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/268d2adf-43d2-4ab0-ae70-f86855843ab9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.861137 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.861115 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-client-ca-bundle\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.861233 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.861209 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-secret-metrics-server-client-certs\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.861346 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.861327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlc4f\" (UniqueName: \"kubernetes.io/projected/268d2adf-43d2-4ab0-ae70-f86855843ab9-kube-api-access-qlc4f\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.861413 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.861371 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/268d2adf-43d2-4ab0-ae70-f86855843ab9-audit-log\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.861413 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.861401 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/268d2adf-43d2-4ab0-ae70-f86855843ab9-metrics-server-audit-profiles\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.861516 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.861464 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-secret-metrics-server-tls\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.861769 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.861747 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/268d2adf-43d2-4ab0-ae70-f86855843ab9-audit-log\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.861917 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.861766 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/268d2adf-43d2-4ab0-ae70-f86855843ab9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.862702 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.862677 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/268d2adf-43d2-4ab0-ae70-f86855843ab9-metrics-server-audit-profiles\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.863904 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.863877 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-secret-metrics-server-tls\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.864018 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.864002 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-client-ca-bundle\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.864077 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.864057 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/268d2adf-43d2-4ab0-ae70-f86855843ab9-secret-metrics-server-client-certs\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:41.872428 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:41.872404 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlc4f\" (UniqueName: \"kubernetes.io/projected/268d2adf-43d2-4ab0-ae70-f86855843ab9-kube-api-access-qlc4f\") pod \"metrics-server-b5f6b554c-46flw\" (UID: \"268d2adf-43d2-4ab0-ae70-f86855843ab9\") " pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:42.037173 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.037037 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:18:42.230588 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.230502 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b5f6b554c-46flw"] Jul 27 14:18:42.233690 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:42.233647 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod268d2adf_43d2_4ab0_ae70_f86855843ab9.slice/crio-f29cab061836b6000d98874cfaef3c3515ef9cff7d55eb646b6d2bbb4e6783ca WatchSource:0}: Error finding container f29cab061836b6000d98874cfaef3c3515ef9cff7d55eb646b6d2bbb4e6783ca: Status 404 returned error can't find the container with id f29cab061836b6000d98874cfaef3c3515ef9cff7d55eb646b6d2bbb4e6783ca Jul 27 14:18:42.631400 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.631318 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7576cf7d6-rrbvb"] Jul 27 14:18:42.645626 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.645590 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.649725 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.649698 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 27 14:18:42.649859 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.649727 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 27 14:18:42.649934 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.649909 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 27 14:18:42.649974 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.649951 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 27 14:18:42.650025 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.649972 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vpzlx\"" Jul 27 14:18:42.650198 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.650185 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 27 14:18:42.655032 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.654955 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7576cf7d6-rrbvb"] Jul 27 14:18:42.656092 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.656069 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 27 14:18:42.669158 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.669125 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-metrics-client-ca\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.669158 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.669156 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6hwd\" (UniqueName: \"kubernetes.io/projected/ffe40e22-cb73-4899-a1e1-190f1ad1a104-kube-api-access-x6hwd\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.669332 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.669199 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-federate-client-tls\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.669332 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.669241 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.669332 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.669268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.669332 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.669291 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-serving-certs-ca-bundle\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.669496 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.669361 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-secret-telemeter-client\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.669496 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.669437 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-telemeter-client-tls\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.770117 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.770080 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-secret-telemeter-client\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.770557 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.770124 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-telemeter-client-tls\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.770557 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.770157 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-metrics-client-ca\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.770557 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.770179 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hwd\" (UniqueName: \"kubernetes.io/projected/ffe40e22-cb73-4899-a1e1-190f1ad1a104-kube-api-access-x6hwd\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.770557 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.770236 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-federate-client-tls\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.770557 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.770300 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.770557 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.770359 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.770557 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.770406 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-serving-certs-ca-bundle\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.771138 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.771114 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-metrics-client-ca\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.771138 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.771126 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-serving-certs-ca-bundle\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.771527 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.771499 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffe40e22-cb73-4899-a1e1-190f1ad1a104-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.772918 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.772896 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.773002 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.772927 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-telemeter-client-tls\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.773002 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.772970 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-secret-telemeter-client\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.773075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.773048 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffe40e22-cb73-4899-a1e1-190f1ad1a104-federate-client-tls\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.780445 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.780421 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6hwd\" (UniqueName: \"kubernetes.io/projected/ffe40e22-cb73-4899-a1e1-190f1ad1a104-kube-api-access-x6hwd\") pod \"telemeter-client-7576cf7d6-rrbvb\" (UID: \"ffe40e22-cb73-4899-a1e1-190f1ad1a104\") " pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:42.861847 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.861806 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" event={"ID":"268d2adf-43d2-4ab0-ae70-f86855843ab9","Type":"ContainerStarted","Data":"f29cab061836b6000d98874cfaef3c3515ef9cff7d55eb646b6d2bbb4e6783ca"} Jul 27 14:18:42.864003 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.863972 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-txldj" event={"ID":"0f366d0f-4f55-4753-be5e-9e781463cc22","Type":"ContainerStarted","Data":"27dea4ec39c5bd33e75099c4b4b799fc71f4cce8c080240429a6df96d65bc71b"} Jul 27 14:18:42.864122 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.864020 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-txldj" event={"ID":"0f366d0f-4f55-4753-be5e-9e781463cc22","Type":"ContainerStarted","Data":"644a217e9d76a7453f558a2e55303896862751a6272bb6c22f7ae53d6e60d864"} Jul 27 14:18:42.897577 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.897457 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-txldj" podStartSLOduration=2.370797763 podStartE2EDuration="5.897437063s" totalImagesPullingTime="3.436967128s" totalInitContainerRuntime="89.672172ms" isStatefulPod=true podCreationTimestamp="2026-07-27 14:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:42.895573255 +0000 UTC m=+106.262835308" watchObservedRunningTime="2026-07-27 14:18:42.897437063 +0000 UTC m=+106.264699112" Jul 27 14:18:42.957585 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:42.957549 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" Jul 27 14:18:43.147925 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:18:43.147847 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe40e22_cb73_4899_a1e1_190f1ad1a104.slice/crio-11a145d887a2b0ffa3a461cbc4f71aa96c65e5e3f3efd04f7ee17d374bf6edf7 WatchSource:0}: Error finding container 11a145d887a2b0ffa3a461cbc4f71aa96c65e5e3f3efd04f7ee17d374bf6edf7: Status 404 returned error can't find the container with id 11a145d887a2b0ffa3a461cbc4f71aa96c65e5e3f3efd04f7ee17d374bf6edf7 Jul 27 14:18:43.148977 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:43.148951 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7576cf7d6-rrbvb"] Jul 27 14:18:43.870773 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:43.870671 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" event={"ID":"ffe40e22-cb73-4899-a1e1-190f1ad1a104","Type":"ContainerStarted","Data":"11a145d887a2b0ffa3a461cbc4f71aa96c65e5e3f3efd04f7ee17d374bf6edf7"} Jul 27 14:18:45.880081 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:45.880001 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" event={"ID":"69f241b0-2404-441a-93f7-97857c9b2da8","Type":"ContainerStarted","Data":"fb9afc254b80fad0f600f63591d82d85a84ba5015a929e28c2457b478c574ebc"} Jul 27 14:18:45.881455 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:45.881427 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" event={"ID":"268d2adf-43d2-4ab0-ae70-f86855843ab9","Type":"ContainerStarted","Data":"4c34a6d2b5e956fc983cfe374c4e21331e1da2debe6110cfeae473322ccbdc56"} Jul 27 14:18:45.903001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:45.902945 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" podStartSLOduration=1.556030287 podStartE2EDuration="4.902926962s" totalImagesPullingTime="3.346896675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:45.902620157 +0000 UTC m=+109.269882230" watchObservedRunningTime="2026-07-27 14:18:45.902926962 +0000 UTC m=+109.270189044" Jul 27 14:18:46.891433 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:46.891394 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" event={"ID":"69f241b0-2404-441a-93f7-97857c9b2da8","Type":"ContainerStarted","Data":"8f64767bc8b313f787011b89795861c8ece1de4f67e67b8b9e538a84bc950759"} Jul 27 14:18:46.891891 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:46.891491 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" event={"ID":"69f241b0-2404-441a-93f7-97857c9b2da8","Type":"ContainerStarted","Data":"bf8e66bdef01025fc0ceb2872ff7b053bad01c8e6b2be3e0668346aeaba8e839"} Jul 27 14:18:46.892898 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:46.892846 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" event={"ID":"ffe40e22-cb73-4899-a1e1-190f1ad1a104","Type":"ContainerStarted","Data":"0d340cb44d3457bb4827b658f2f2b1240896b12457679d4227bdcf6186f9a15b"} Jul 27 14:18:47.898550 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:47.898470 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" event={"ID":"ffe40e22-cb73-4899-a1e1-190f1ad1a104","Type":"ContainerStarted","Data":"4810bac6f68a600f1c0849deabf6a777611eb6793d67bd6d3b2ed5b538a52bb2"} Jul 27 14:18:47.898550 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:47.898529 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" event={"ID":"ffe40e22-cb73-4899-a1e1-190f1ad1a104","Type":"ContainerStarted","Data":"8bb7c78a88fb71c7d8c959ef00ffd493ca523e3008e7d5ac85b7df7fed5a1645"} Jul 27 14:18:47.929035 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:47.928978 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7576cf7d6-rrbvb" podStartSLOduration=1.770423976 podStartE2EDuration="5.928958067s" totalImagesPullingTime="4.158534091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:18:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:47.926748632 +0000 UTC m=+111.294010680" watchObservedRunningTime="2026-07-27 14:18:47.928958067 +0000 UTC m=+111.296220114" Jul 27 14:18:49.909471 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:49.909435 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" event={"ID":"69f241b0-2404-441a-93f7-97857c9b2da8","Type":"ContainerStarted","Data":"aedb24a724a807de9e54e27b67062f2aa695442cfb23ffa45cc1514af331968e"} Jul 27 14:18:49.909947 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:49.909507 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" event={"ID":"69f241b0-2404-441a-93f7-97857c9b2da8","Type":"ContainerStarted","Data":"185a8afc085339d8ff312227fd27a531e1b1f8a393c3e8b3c76543694feaffb7"} Jul 27 14:18:49.909947 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:49.909534 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" event={"ID":"69f241b0-2404-441a-93f7-97857c9b2da8","Type":"ContainerStarted","Data":"6f1c59feb5a3e5206cc2d5e98695b543af11a16d2860032dd49821fc19545a16"} Jul 27 14:18:49.909947 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:49.909721 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:49.945036 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:49.944986 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" podStartSLOduration=2.345421243 podStartE2EDuration="9.944969019s" totalImagesPullingTime="7.599547776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:18:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:18:49.943421563 +0000 UTC m=+113.310683611" watchObservedRunningTime="2026-07-27 14:18:49.944969019 +0000 UTC m=+113.312231065" Jul 27 14:18:50.919460 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:50.919434 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5dfbfbd66c-pfzwf" Jul 27 14:18:54.931918 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:54.931886 2555 generic.go:356] "Generic (PLEG): container finished" podID="76fb3e7b-bef0-473d-a98e-0cbac98c891c" containerID="56cde6dcdf21abeed69d59d2ff152ae81b15df647b0a407d5adc1e9b23a29a71" exitCode=0 Jul 27 14:18:54.932334 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:54.931962 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wdpj7" event={"ID":"76fb3e7b-bef0-473d-a98e-0cbac98c891c","Type":"ContainerDied","Data":"56cde6dcdf21abeed69d59d2ff152ae81b15df647b0a407d5adc1e9b23a29a71"} Jul 27 14:18:54.932376 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:54.932357 2555 scope.go:117] "RemoveContainer" containerID="56cde6dcdf21abeed69d59d2ff152ae81b15df647b0a407d5adc1e9b23a29a71" Jul 27 14:18:55.937035 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:55.937000 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wdpj7" event={"ID":"76fb3e7b-bef0-473d-a98e-0cbac98c891c","Type":"ContainerStarted","Data":"9fabbe01341d6218ee0791941c6ae22c2bb7e1e23a324c8efae2fffa95972b63"} Jul 27 14:18:57.094902 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.094846 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" podUID="3516fbc8-37ce-4433-bbff-4faea6399efe" containerName="registry" containerID="cri-o://0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254" gracePeriod=30 Jul 27 14:18:57.341069 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.341043 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:18:57.402299 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402226 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3516fbc8-37ce-4433-bbff-4faea6399efe-ca-trust-extracted\") pod \"3516fbc8-37ce-4433-bbff-4faea6399efe\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " Jul 27 14:18:57.402299 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402271 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-trusted-ca\") pod \"3516fbc8-37ce-4433-bbff-4faea6399efe\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " Jul 27 14:18:57.402299 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402290 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-certificates\") pod \"3516fbc8-37ce-4433-bbff-4faea6399efe\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " Jul 27 14:18:57.402555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402327 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-installation-pull-secrets\") pod \"3516fbc8-37ce-4433-bbff-4faea6399efe\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " Jul 27 14:18:57.402555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402369 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcdtm\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-kube-api-access-dcdtm\") pod \"3516fbc8-37ce-4433-bbff-4faea6399efe\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " Jul 27 14:18:57.402555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402392 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-image-registry-private-configuration\") pod \"3516fbc8-37ce-4433-bbff-4faea6399efe\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " Jul 27 14:18:57.402555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402428 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-bound-sa-token\") pod \"3516fbc8-37ce-4433-bbff-4faea6399efe\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " Jul 27 14:18:57.402555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402524 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") pod \"3516fbc8-37ce-4433-bbff-4faea6399efe\" (UID: \"3516fbc8-37ce-4433-bbff-4faea6399efe\") " Jul 27 14:18:57.402829 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402702 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3516fbc8-37ce-4433-bbff-4faea6399efe" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 14:18:57.402889 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402857 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3516fbc8-37ce-4433-bbff-4faea6399efe" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 14:18:57.402889 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.402868 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-trusted-ca\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:18:57.404918 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.404890 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3516fbc8-37ce-4433-bbff-4faea6399efe" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:18:57.405044 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.404974 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3516fbc8-37ce-4433-bbff-4faea6399efe" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:18:57.405106 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.405089 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3516fbc8-37ce-4433-bbff-4faea6399efe" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:18:57.405146 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.405090 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-kube-api-access-dcdtm" (OuterVolumeSpecName: "kube-api-access-dcdtm") pod "3516fbc8-37ce-4433-bbff-4faea6399efe" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe"). InnerVolumeSpecName "kube-api-access-dcdtm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:18:57.405195 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.405178 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3516fbc8-37ce-4433-bbff-4faea6399efe" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:18:57.411054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.411027 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3516fbc8-37ce-4433-bbff-4faea6399efe-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3516fbc8-37ce-4433-bbff-4faea6399efe" (UID: "3516fbc8-37ce-4433-bbff-4faea6399efe"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:18:57.503842 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.503815 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-installation-pull-secrets\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:18:57.503842 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.503838 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcdtm\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-kube-api-access-dcdtm\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:18:57.503842 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.503848 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3516fbc8-37ce-4433-bbff-4faea6399efe-image-registry-private-configuration\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:18:57.504069 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.503859 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-bound-sa-token\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:18:57.504069 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.503868 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:18:57.504069 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.503876 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3516fbc8-37ce-4433-bbff-4faea6399efe-ca-trust-extracted\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:18:57.504069 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.503886 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3516fbc8-37ce-4433-bbff-4faea6399efe-registry-certificates\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:18:57.945828 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.945792 2555 generic.go:356] "Generic (PLEG): container finished" podID="3516fbc8-37ce-4433-bbff-4faea6399efe" containerID="0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254" exitCode=0 Jul 27 14:18:57.946000 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.945838 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" Jul 27 14:18:57.946000 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.945864 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" event={"ID":"3516fbc8-37ce-4433-bbff-4faea6399efe","Type":"ContainerDied","Data":"0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254"} Jul 27 14:18:57.946000 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.945912 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f6585d46-wlhhh" event={"ID":"3516fbc8-37ce-4433-bbff-4faea6399efe","Type":"ContainerDied","Data":"616e3526e5b9751700acd724115bf162bcbab0dee22ba7c61dfa8d42c400e09b"} Jul 27 14:18:57.946000 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.945938 2555 scope.go:117] "RemoveContainer" containerID="0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254" Jul 27 14:18:57.954784 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.954765 2555 scope.go:117] "RemoveContainer" containerID="0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254" Jul 27 14:18:57.955029 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:18:57.955009 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254\": container with ID starting with 0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254 not found: ID does not exist" containerID="0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254" Jul 27 14:18:57.955098 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.955044 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254"} err="failed to get container status \"0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254\": rpc error: code = NotFound desc = could not find container \"0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254\": container with ID starting with 0738d2bb2473bd2e6da0ab2737efdf32697a1bb61ec6d0d7a5235fdd467c8254 not found: ID does not exist" Jul 27 14:18:57.967036 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.967013 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76f6585d46-wlhhh"] Jul 27 14:18:57.978440 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:57.978413 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76f6585d46-wlhhh"] Jul 27 14:18:59.146835 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:18:59.146802 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3516fbc8-37ce-4433-bbff-4faea6399efe" path="/var/lib/kubelet/pods/3516fbc8-37ce-4433-bbff-4faea6399efe/volumes" Jul 27 14:19:02.038101 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:02.038060 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:19:02.038611 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:02.038138 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:19:15.017284 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:15.017249 2555 generic.go:356] "Generic (PLEG): container finished" podID="a9ba173c-70f2-4c02-a10d-f48753f7f720" containerID="14b53bd611d3e3991c26c8bbfb6a6b3d7bce3467d501142f830ec089ee894ed7" exitCode=0 Jul 27 14:19:15.017713 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:15.017321 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" event={"ID":"a9ba173c-70f2-4c02-a10d-f48753f7f720","Type":"ContainerDied","Data":"14b53bd611d3e3991c26c8bbfb6a6b3d7bce3467d501142f830ec089ee894ed7"} Jul 27 14:19:15.017713 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:15.017649 2555 scope.go:117] "RemoveContainer" containerID="14b53bd611d3e3991c26c8bbfb6a6b3d7bce3467d501142f830ec089ee894ed7" Jul 27 14:19:16.023305 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:16.023264 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q9dvs" event={"ID":"a9ba173c-70f2-4c02-a10d-f48753f7f720","Type":"ContainerStarted","Data":"7c6c2e4ff3dc158af8b249fc80f96f13e5d9aad405afc2d5c7a33c63a7761800"} Jul 27 14:19:16.024763 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:16.024742 2555 generic.go:356] "Generic (PLEG): container finished" podID="f86b19c8-ef8a-4e72-8048-c9fe8112d3ec" containerID="9afa26bc25c189ae1979339ba64adb715e6abcd9b2e442cc6c448c588d3a484d" exitCode=0 Jul 27 14:19:16.024857 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:16.024785 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" event={"ID":"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec","Type":"ContainerDied","Data":"9afa26bc25c189ae1979339ba64adb715e6abcd9b2e442cc6c448c588d3a484d"} Jul 27 14:19:16.025031 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:16.025019 2555 scope.go:117] "RemoveContainer" containerID="9afa26bc25c189ae1979339ba64adb715e6abcd9b2e442cc6c448c588d3a484d" Jul 27 14:19:17.030521 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:17.030487 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s" event={"ID":"f86b19c8-ef8a-4e72-8048-c9fe8112d3ec","Type":"ContainerStarted","Data":"6cd062dc013d3ab20515282505814d086e97376361fb82799e7097b88d2dbbe2"} Jul 27 14:19:22.043018 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:22.042983 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:19:22.046833 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:19:22.046808 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b5f6b554c-46flw" Jul 27 14:20:00.573105 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.573076 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 14:20:00.573528 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.573383 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3516fbc8-37ce-4433-bbff-4faea6399efe" containerName="registry" Jul 27 14:20:00.573528 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.573402 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3516fbc8-37ce-4433-bbff-4faea6399efe" containerName="registry" Jul 27 14:20:00.573528 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.573472 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3516fbc8-37ce-4433-bbff-4faea6399efe" containerName="registry" Jul 27 14:20:00.578390 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.578373 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.581049 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581025 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 27 14:20:00.581169 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581025 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j9v9k\"" Jul 27 14:20:00.581253 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581223 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 27 14:20:00.581382 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581362 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 27 14:20:00.581473 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581376 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 27 14:20:00.581473 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581381 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 27 14:20:00.581473 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581370 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 27 14:20:00.581639 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581409 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 27 14:20:00.581724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.581676 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 27 14:20:00.587616 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.587597 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 27 14:20:00.592292 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.592274 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 14:20:00.739743 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739710 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6e8899c-f0ac-49ce-be97-ff98b9e55393-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.739743 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739747 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6e8899c-f0ac-49ce-be97-ff98b9e55393-config-out\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.739951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739784 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.739951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739811 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6e8899c-f0ac-49ce-be97-ff98b9e55393-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.739951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739833 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-web-config\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.739951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739888 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.739951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739920 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.739951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739945 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24pbv\" (UniqueName: \"kubernetes.io/projected/c6e8899c-f0ac-49ce-be97-ff98b9e55393-kube-api-access-24pbv\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.740137 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.739988 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6e8899c-f0ac-49ce-be97-ff98b9e55393-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.740137 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.740009 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-config-volume\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.740137 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.740033 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c6e8899c-f0ac-49ce-be97-ff98b9e55393-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.740137 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.740063 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.740137 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.740083 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841456 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6e8899c-f0ac-49ce-be97-ff98b9e55393-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841490 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-config-volume\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841512 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c6e8899c-f0ac-49ce-be97-ff98b9e55393-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841548 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841573 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841609 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6e8899c-f0ac-49ce-be97-ff98b9e55393-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6e8899c-f0ac-49ce-be97-ff98b9e55393-config-out\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841692 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841720 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6e8899c-f0ac-49ce-be97-ff98b9e55393-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841744 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-web-config\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841782 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.841820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841806 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.842246 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.841831 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24pbv\" (UniqueName: \"kubernetes.io/projected/c6e8899c-f0ac-49ce-be97-ff98b9e55393-kube-api-access-24pbv\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.842301 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.842257 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c6e8899c-f0ac-49ce-be97-ff98b9e55393-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.842355 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.842321 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6e8899c-f0ac-49ce-be97-ff98b9e55393-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.843555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.843526 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6e8899c-f0ac-49ce-be97-ff98b9e55393-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.844805 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.844775 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.844919 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.844879 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-config-volume\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.844919 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.844907 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.845028 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.844920 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6e8899c-f0ac-49ce-be97-ff98b9e55393-config-out\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.845028 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.844965 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.845200 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.845183 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6e8899c-f0ac-49ce-be97-ff98b9e55393-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.845272 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.845254 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-web-config\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.846117 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.846098 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.846731 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.846713 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c6e8899c-f0ac-49ce-be97-ff98b9e55393-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.849745 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.849650 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24pbv\" (UniqueName: \"kubernetes.io/projected/c6e8899c-f0ac-49ce-be97-ff98b9e55393-kube-api-access-24pbv\") pod \"alertmanager-main-0\" (UID: \"c6e8899c-f0ac-49ce-be97-ff98b9e55393\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:00.887737 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:00.887712 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 14:20:01.022190 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:01.022167 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 14:20:01.025143 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:20:01.025114 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e8899c_f0ac_49ce_be97_ff98b9e55393.slice/crio-65ab8885ea680c2dff25f55052f9c1da88a85ee0990f2bfbfe25b944a80d6760 WatchSource:0}: Error finding container 65ab8885ea680c2dff25f55052f9c1da88a85ee0990f2bfbfe25b944a80d6760: Status 404 returned error can't find the container with id 65ab8885ea680c2dff25f55052f9c1da88a85ee0990f2bfbfe25b944a80d6760 Jul 27 14:20:01.196830 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:01.196798 2555 generic.go:356] "Generic (PLEG): container finished" podID="c6e8899c-f0ac-49ce-be97-ff98b9e55393" containerID="4d813df3b66fe1d04057fd2ec2c5bfceb3056e80e7b26a3083911e68b62982c7" exitCode=0 Jul 27 14:20:01.196996 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:01.196887 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c6e8899c-f0ac-49ce-be97-ff98b9e55393","Type":"ContainerDied","Data":"4d813df3b66fe1d04057fd2ec2c5bfceb3056e80e7b26a3083911e68b62982c7"} Jul 27 14:20:01.196996 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:01.196973 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c6e8899c-f0ac-49ce-be97-ff98b9e55393","Type":"ContainerStarted","Data":"65ab8885ea680c2dff25f55052f9c1da88a85ee0990f2bfbfe25b944a80d6760"} Jul 27 14:20:06.218676 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:06.218626 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c6e8899c-f0ac-49ce-be97-ff98b9e55393","Type":"ContainerStarted","Data":"6f190282c29588d508e9ee50a4ba8227fb0368276f5a625b39f1eac30adf09bb"} Jul 27 14:20:06.218972 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:06.218706 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c6e8899c-f0ac-49ce-be97-ff98b9e55393","Type":"ContainerStarted","Data":"108e239cc6afeede7b15a8ef365a36cbf996d51923e7a09fb1ca0b9601893c78"} Jul 27 14:20:06.218972 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:06.218736 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c6e8899c-f0ac-49ce-be97-ff98b9e55393","Type":"ContainerStarted","Data":"b9d4970cb9ff31fbb6bea9f4c53939ec1ce58a7c694b1806053ec08d2b4f155a"} Jul 27 14:20:07.225312 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:07.225273 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c6e8899c-f0ac-49ce-be97-ff98b9e55393","Type":"ContainerStarted","Data":"c42173cb80872a1384551da8e221b82dd8ceb6daa6151c29d529f3487599741a"} Jul 27 14:20:07.225723 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:07.225352 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c6e8899c-f0ac-49ce-be97-ff98b9e55393","Type":"ContainerStarted","Data":"5a8c479054a20a3a903179e2cb69251bf50045ae46458acab35b415bd2fccf9c"} Jul 27 14:20:07.225723 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:07.225393 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c6e8899c-f0ac-49ce-be97-ff98b9e55393","Type":"ContainerStarted","Data":"d13c20ab7f64f44d2b9e046902dbf3d255c18148664c55c966163271c977174a"} Jul 27 14:20:07.269694 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:20:07.269614 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.469063512 podStartE2EDuration="7.269593208s" totalImagesPullingTime="4.784040301s" totalInitContainerRuntime="16.489395ms" isStatefulPod=false podCreationTimestamp="2026-07-27 14:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:20:07.267181808 +0000 UTC m=+190.634443857" watchObservedRunningTime="2026-07-27 14:20:07.269593208 +0000 UTC m=+190.636855256" Jul 27 14:21:56.995114 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:21:56.995085 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:21:57.003147 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:21:57.003119 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:21:57.006761 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:21:57.006740 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 27 14:23:21.429907 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.429872 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw"] Jul 27 14:23:21.433193 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.433173 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:23:21.435947 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.435926 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jul 27 14:23:21.436054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.435926 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jul 27 14:23:21.436912 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.436897 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-qsfzr\"" Jul 27 14:23:21.446001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.445976 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw"] Jul 27 14:23:21.523361 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.523320 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg7nr\" (UniqueName: \"kubernetes.io/projected/d54ce285-0716-426f-9c96-ce2ba6cab7be-kube-api-access-zg7nr\") pod \"feast-operator-controller-manager-6598b8cfdf-8ngdw\" (UID: \"d54ce285-0716-426f-9c96-ce2ba6cab7be\") " pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:23:21.623819 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.623785 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg7nr\" (UniqueName: \"kubernetes.io/projected/d54ce285-0716-426f-9c96-ce2ba6cab7be-kube-api-access-zg7nr\") pod \"feast-operator-controller-manager-6598b8cfdf-8ngdw\" (UID: \"d54ce285-0716-426f-9c96-ce2ba6cab7be\") " pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:23:21.632380 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.632351 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg7nr\" (UniqueName: \"kubernetes.io/projected/d54ce285-0716-426f-9c96-ce2ba6cab7be-kube-api-access-zg7nr\") pod \"feast-operator-controller-manager-6598b8cfdf-8ngdw\" (UID: \"d54ce285-0716-426f-9c96-ce2ba6cab7be\") " pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:23:21.743821 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.743738 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:23:21.874509 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.874482 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw"] Jul 27 14:23:21.876603 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:23:21.876578 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54ce285_0716_426f_9c96_ce2ba6cab7be.slice/crio-a3db96585fa4c57da4c205a1d1fc9e3e8205f12aa19bd6882c62955812b96df2 WatchSource:0}: Error finding container a3db96585fa4c57da4c205a1d1fc9e3e8205f12aa19bd6882c62955812b96df2: Status 404 returned error can't find the container with id a3db96585fa4c57da4c205a1d1fc9e3e8205f12aa19bd6882c62955812b96df2 Jul 27 14:23:21.878595 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:21.878578 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 14:23:22.013356 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:22.013270 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" event={"ID":"d54ce285-0716-426f-9c96-ce2ba6cab7be","Type":"ContainerStarted","Data":"a3db96585fa4c57da4c205a1d1fc9e3e8205f12aa19bd6882c62955812b96df2"} Jul 27 14:23:25.030622 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:25.030582 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" event={"ID":"d54ce285-0716-426f-9c96-ce2ba6cab7be","Type":"ContainerStarted","Data":"29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f"} Jul 27 14:23:25.031142 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:25.030767 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:23:25.047546 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:25.047496 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" podStartSLOduration=1.792367397 podStartE2EDuration="4.047482269s" totalImagesPullingTime="2.255114872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:23:25.046794374 +0000 UTC m=+388.414056435" watchObservedRunningTime="2026-07-27 14:23:25.047482269 +0000 UTC m=+388.414744318" Jul 27 14:23:36.037145 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:36.037117 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:23:57.689178 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.689145 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n"] Jul 27 14:23:57.695961 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.695941 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.699154 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.699130 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 27 14:23:57.699976 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.699943 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 27 14:23:57.699976 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.699962 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 27 14:23:57.700158 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.699952 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jul 27 14:23:57.700158 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.699954 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-bs2f8\"" Jul 27 14:23:57.700158 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.699968 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 27 14:23:57.700158 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.699951 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 27 14:23:57.700359 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.700209 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 27 14:23:57.704139 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.704117 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n"] Jul 27 14:23:57.844294 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.844252 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-registry-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.844468 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.844378 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-ui-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.844468 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.844414 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqjwf\" (UniqueName: \"kubernetes.io/projected/970c35db-69b6-414d-a853-6ad124b7f244-kube-api-access-gqjwf\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.844468 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.844451 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/970c35db-69b6-414d-a853-6ad124b7f244-feast-data\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.844468 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.844468 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-offline-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.844628 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.844551 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-online-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.945499 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.945405 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-registry-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.945499 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.945489 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-ui-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.945786 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.945509 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqjwf\" (UniqueName: \"kubernetes.io/projected/970c35db-69b6-414d-a853-6ad124b7f244-kube-api-access-gqjwf\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.945786 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.945537 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/970c35db-69b6-414d-a853-6ad124b7f244-feast-data\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.945786 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.945559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-offline-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.945786 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.945580 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-online-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.946062 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.946035 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/970c35db-69b6-414d-a853-6ad124b7f244-feast-data\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.948131 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.948103 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-ui-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.948230 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.948164 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-online-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.948270 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.948246 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-registry-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.948270 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.948252 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-offline-tls\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:57.954352 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:57.954321 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqjwf\" (UniqueName: \"kubernetes.io/projected/970c35db-69b6-414d-a853-6ad124b7f244-kube-api-access-gqjwf\") pod \"feast-simple-feast-setup-5479df6db8-z5x6n\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:58.020426 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:58.020372 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:23:58.168972 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:58.168942 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n"] Jul 27 14:23:58.171617 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:23:58.171591 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod970c35db_69b6_414d_a853_6ad124b7f244.slice/crio-06f0246a2bfecc599fe4bc78ea14615fbe219d03b6d25b68e8b6a9ecfa24cdbb WatchSource:0}: Error finding container 06f0246a2bfecc599fe4bc78ea14615fbe219d03b6d25b68e8b6a9ecfa24cdbb: Status 404 returned error can't find the container with id 06f0246a2bfecc599fe4bc78ea14615fbe219d03b6d25b68e8b6a9ecfa24cdbb Jul 27 14:23:59.179127 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:23:59.179088 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerStarted","Data":"06f0246a2bfecc599fe4bc78ea14615fbe219d03b6d25b68e8b6a9ecfa24cdbb"} Jul 27 14:24:23.292534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:23.292497 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerStarted","Data":"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22"} Jul 27 14:24:27.314023 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:27.313989 2555 generic.go:356] "Generic (PLEG): container finished" podID="970c35db-69b6-414d-a853-6ad124b7f244" containerID="38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22" exitCode=0 Jul 27 14:24:27.314390 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:27.314050 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerDied","Data":"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22"} Jul 27 14:24:28.320066 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:28.320028 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerStarted","Data":"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f"} Jul 27 14:24:33.340179 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:33.340142 2555 generic.go:356] "Generic (PLEG): container finished" podID="970c35db-69b6-414d-a853-6ad124b7f244" containerID="5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f" exitCode=0 Jul 27 14:24:33.340678 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:33.340214 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerDied","Data":"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f"} Jul 27 14:24:34.351595 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:34.351551 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerStarted","Data":"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f"} Jul 27 14:24:34.352102 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:34.351616 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerStarted","Data":"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557"} Jul 27 14:24:34.352102 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:34.351634 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerStarted","Data":"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2"} Jul 27 14:24:34.352102 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:34.351682 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerStarted","Data":"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b"} Jul 27 14:24:34.378852 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:34.378778 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podStartSLOduration=13.095015897 podStartE2EDuration="37.378756722s" totalImagesPullingTime="24.283740825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:24:34.37625096 +0000 UTC m=+457.743513009" watchObservedRunningTime="2026-07-27 14:24:34.378756722 +0000 UTC m=+457.746018770" Jul 27 14:24:37.021539 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:37.021479 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:37.022163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:37.022138 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:37.022300 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:37.022176 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:37.022300 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:37.022195 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:37.023532 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:37.023490 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused" Jul 27 14:24:37.023532 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:37.023509 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.25:6571: connect: connection refused" Jul 27 14:24:37.023727 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:37.023499 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.25:8816: connect: connection refused" Jul 27 14:24:37.023727 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:37.023555 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused" Jul 27 14:24:38.020541 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:38.020494 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:38.020783 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:38.020566 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:38.020783 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:38.020589 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:38.020783 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:38.020608 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:40.021228 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:40.021193 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:40.021768 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:40.021328 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:40.021827 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:40.021811 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:40.022298 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:40.022278 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:40.022793 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:40.022778 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:40.027490 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:40.027473 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:40.386555 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:40.386485 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:24:40.389166 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:24:40.389147 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:25:42.953110 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:42.953073 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n"] Jul 27 14:25:42.953737 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:42.953390 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="registry" containerID="cri-o://9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b" gracePeriod=30 Jul 27 14:25:42.953737 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:42.953621 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="ui" containerID="cri-o://4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f" gracePeriod=30 Jul 27 14:25:42.953894 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:42.953732 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="offline" containerID="cri-o://efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557" gracePeriod=30 Jul 27 14:25:42.953894 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:42.953734 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="online" containerID="cri-o://9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2" gracePeriod=30 Jul 27 14:25:43.241176 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.241086 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp"] Jul 27 14:25:43.245350 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.245328 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.248220 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.248186 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-mmpzp\"" Jul 27 14:25:43.256624 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.256597 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp"] Jul 27 14:25:43.381713 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.381643 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.381913 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.381753 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.381913 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.381800 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-registry-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.381913 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.381826 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-offline-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.382083 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.381912 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0645d16f-fc2f-4e72-af20-cd778dfdc28c-feast-data\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.382083 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.382020 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk4lr\" (UniqueName: \"kubernetes.io/projected/0645d16f-fc2f-4e72-af20-cd778dfdc28c-kube-api-access-qk4lr\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.482718 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.482684 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk4lr\" (UniqueName: \"kubernetes.io/projected/0645d16f-fc2f-4e72-af20-cd778dfdc28c-kube-api-access-qk4lr\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.482904 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.482765 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.482904 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.482791 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.482904 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.482819 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-registry-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.482904 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.482840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-offline-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.482904 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.482883 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0645d16f-fc2f-4e72-af20-cd778dfdc28c-feast-data\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.483173 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:25:43.482931 2555 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jul 27 14:25:43.483173 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:25:43.482931 2555 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jul 27 14:25:43.483173 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:25:43.483004 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls podName:0645d16f-fc2f-4e72-af20-cd778dfdc28c nodeName:}" failed. No retries permitted until 2026-07-27 14:25:43.982981366 +0000 UTC m=+527.350243407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls") pod "feast-simple-feast-setup-5479df6db8-pt7sp" (UID: "0645d16f-fc2f-4e72-af20-cd778dfdc28c") : secret "feast-simple-feast-setup-online-tls" not found Jul 27 14:25:43.483173 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:25:43.483089 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls podName:0645d16f-fc2f-4e72-af20-cd778dfdc28c nodeName:}" failed. No retries permitted until 2026-07-27 14:25:43.983049293 +0000 UTC m=+527.350311318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls") pod "feast-simple-feast-setup-5479df6db8-pt7sp" (UID: "0645d16f-fc2f-4e72-af20-cd778dfdc28c") : secret "feast-simple-feast-setup-ui-tls" not found Jul 27 14:25:43.483331 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.483297 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0645d16f-fc2f-4e72-af20-cd778dfdc28c-feast-data\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.485492 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.485459 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-offline-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.486238 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.486216 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-registry-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.491100 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.491071 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk4lr\" (UniqueName: \"kubernetes.io/projected/0645d16f-fc2f-4e72-af20-cd778dfdc28c-kube-api-access-qk4lr\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.661598 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.661562 2555 generic.go:356] "Generic (PLEG): container finished" podID="970c35db-69b6-414d-a853-6ad124b7f244" containerID="4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f" exitCode=0 Jul 27 14:25:43.661773 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.661605 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerDied","Data":"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f"} Jul 27 14:25:43.987443 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.987407 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.987845 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.987449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.989869 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.989844 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:43.989869 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:43.989860 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls\") pod \"feast-simple-feast-setup-5479df6db8-pt7sp\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:44.155487 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:44.155447 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:44.288906 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:44.288863 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp"] Jul 27 14:25:44.289544 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:25:44.289518 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0645d16f_fc2f_4e72_af20_cd778dfdc28c.slice/crio-42c555b39b9b09b5949f522ba4a18cbed0d663a8ccde931561efed2af349364a WatchSource:0}: Error finding container 42c555b39b9b09b5949f522ba4a18cbed0d663a8ccde931561efed2af349364a: Status 404 returned error can't find the container with id 42c555b39b9b09b5949f522ba4a18cbed0d663a8ccde931561efed2af349364a Jul 27 14:25:44.668073 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:44.667974 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerStarted","Data":"9aa067e76d7cf7a0cb7aa6f115aead33046cd6ccf8ff1590b2579173ff6f11e4"} Jul 27 14:25:44.668073 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:44.668026 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerStarted","Data":"42c555b39b9b09b5949f522ba4a18cbed0d663a8ccde931561efed2af349364a"} Jul 27 14:25:44.670905 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:44.670878 2555 generic.go:356] "Generic (PLEG): container finished" podID="970c35db-69b6-414d-a853-6ad124b7f244" containerID="9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2" exitCode=0 Jul 27 14:25:44.671021 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:44.670950 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerDied","Data":"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2"} Jul 27 14:25:49.694337 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:49.694305 2555 generic.go:356] "Generic (PLEG): container finished" podID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerID="9aa067e76d7cf7a0cb7aa6f115aead33046cd6ccf8ff1590b2579173ff6f11e4" exitCode=0 Jul 27 14:25:49.694731 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:49.694370 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerDied","Data":"9aa067e76d7cf7a0cb7aa6f115aead33046cd6ccf8ff1590b2579173ff6f11e4"} Jul 27 14:25:50.386082 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:50.386034 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused" Jul 27 14:25:50.386307 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:50.386042 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused" Jul 27 14:25:50.700417 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:50.700378 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerStarted","Data":"7382f07c44dfb143a825d6f3dd39113a0872e782df129b1548c07844c28119c8"} Jul 27 14:25:56.730782 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:56.730751 2555 generic.go:356] "Generic (PLEG): container finished" podID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerID="7382f07c44dfb143a825d6f3dd39113a0872e782df129b1548c07844c28119c8" exitCode=0 Jul 27 14:25:56.731171 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:56.730827 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerDied","Data":"7382f07c44dfb143a825d6f3dd39113a0872e782df129b1548c07844c28119c8"} Jul 27 14:25:57.741401 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:57.741350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerStarted","Data":"849c6e77ca18f8c556582f7037ff359a962d185ec6e5df45022a53670a70b3c0"} Jul 27 14:25:57.741954 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:57.741423 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerStarted","Data":"a781da4660b30fd8863917b6ff416c761f6f62406d350e8f9f937097ac403bef"} Jul 27 14:25:57.741954 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:57.741446 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerStarted","Data":"7e63082ff260211ae3af5eda4f43752bf52478c64258549498f258e876c939ca"} Jul 27 14:25:57.741954 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:57.741464 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerStarted","Data":"3c379c057c094effd9a6f35c802f8d8f300b72aedae10597dee3918647e2bab4"} Jul 27 14:25:57.770015 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:57.769949 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podStartSLOduration=14.769926227 podStartE2EDuration="14.769926227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:25:57.76760013 +0000 UTC m=+541.134862177" watchObservedRunningTime="2026-07-27 14:25:57.769926227 +0000 UTC m=+541.137188276" Jul 27 14:25:59.156715 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:59.156577 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:59.156715 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:59.156635 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:59.156715 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:59.156669 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:59.156715 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:59.156689 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:25:59.158605 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:59.158566 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.26:8816: connect: connection refused" Jul 27 14:25:59.158865 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:59.158588 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Jul 27 14:25:59.158951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:59.158606 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 27 14:25:59.158951 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:25:59.158699 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 27 14:26:00.386534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:00.386484 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused" Jul 27 14:26:00.387077 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:00.386543 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused" Jul 27 14:26:02.156651 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:02.156602 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.26:8816: connect: connection refused" Jul 27 14:26:02.157423 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:02.156606 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 27 14:26:02.157423 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:02.156627 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Jul 27 14:26:02.157423 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:02.156673 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 27 14:26:04.156335 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:04.156298 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:04.156745 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:04.156351 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:04.156745 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:04.156367 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:04.156745 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:04.156376 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:05.156600 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:05.156567 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:05.157161 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:05.156879 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:05.157472 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:05.157450 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:05.157636 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:05.157508 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:05.157636 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:05.157540 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:05.162194 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:05.162172 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:05.785219 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:05.785194 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:05.787693 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:05.787677 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:26:10.386072 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:10.386021 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused" Jul 27 14:26:10.386491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:10.386033 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused" Jul 27 14:26:10.386491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:10.386184 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:26:10.386491 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:10.386249 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:26:13.619522 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.619494 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:26:13.660906 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.660869 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/970c35db-69b6-414d-a853-6ad124b7f244-feast-data\") pod \"970c35db-69b6-414d-a853-6ad124b7f244\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " Jul 27 14:26:13.661075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.660938 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-online-tls\") pod \"970c35db-69b6-414d-a853-6ad124b7f244\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " Jul 27 14:26:13.661075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.660975 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-registry-tls\") pod \"970c35db-69b6-414d-a853-6ad124b7f244\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " Jul 27 14:26:13.661075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.661018 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-offline-tls\") pod \"970c35db-69b6-414d-a853-6ad124b7f244\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " Jul 27 14:26:13.661295 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.661103 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqjwf\" (UniqueName: \"kubernetes.io/projected/970c35db-69b6-414d-a853-6ad124b7f244-kube-api-access-gqjwf\") pod \"970c35db-69b6-414d-a853-6ad124b7f244\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " Jul 27 14:26:13.661295 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.661152 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-ui-tls\") pod \"970c35db-69b6-414d-a853-6ad124b7f244\" (UID: \"970c35db-69b6-414d-a853-6ad124b7f244\") " Jul 27 14:26:13.661495 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.661466 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970c35db-69b6-414d-a853-6ad124b7f244-feast-data" (OuterVolumeSpecName: "feast-data") pod "970c35db-69b6-414d-a853-6ad124b7f244" (UID: "970c35db-69b6-414d-a853-6ad124b7f244"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:26:13.663329 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.663300 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "970c35db-69b6-414d-a853-6ad124b7f244" (UID: "970c35db-69b6-414d-a853-6ad124b7f244"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:26:13.663429 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.663323 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-online-tls" (OuterVolumeSpecName: "online-tls") pod "970c35db-69b6-414d-a853-6ad124b7f244" (UID: "970c35db-69b6-414d-a853-6ad124b7f244"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:26:13.663521 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.663504 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970c35db-69b6-414d-a853-6ad124b7f244-kube-api-access-gqjwf" (OuterVolumeSpecName: "kube-api-access-gqjwf") pod "970c35db-69b6-414d-a853-6ad124b7f244" (UID: "970c35db-69b6-414d-a853-6ad124b7f244"). InnerVolumeSpecName "kube-api-access-gqjwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:26:13.663652 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.663632 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "970c35db-69b6-414d-a853-6ad124b7f244" (UID: "970c35db-69b6-414d-a853-6ad124b7f244"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:26:13.663723 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.663647 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "970c35db-69b6-414d-a853-6ad124b7f244" (UID: "970c35db-69b6-414d-a853-6ad124b7f244"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:26:13.762724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.762629 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqjwf\" (UniqueName: \"kubernetes.io/projected/970c35db-69b6-414d-a853-6ad124b7f244-kube-api-access-gqjwf\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:26:13.762724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.762693 2555 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-ui-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:26:13.762724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.762703 2555 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/970c35db-69b6-414d-a853-6ad124b7f244-feast-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:26:13.762724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.762714 2555 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-online-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:26:13.762724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.762722 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-registry-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:26:13.762724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.762729 2555 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/970c35db-69b6-414d-a853-6ad124b7f244-offline-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:26:13.821937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.821903 2555 generic.go:356] "Generic (PLEG): container finished" podID="970c35db-69b6-414d-a853-6ad124b7f244" containerID="efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557" exitCode=137 Jul 27 14:26:13.821937 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.821930 2555 generic.go:356] "Generic (PLEG): container finished" podID="970c35db-69b6-414d-a853-6ad124b7f244" containerID="9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b" exitCode=137 Jul 27 14:26:13.822146 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.821967 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" Jul 27 14:26:13.822146 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.821972 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerDied","Data":"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557"} Jul 27 14:26:13.822146 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.822008 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerDied","Data":"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b"} Jul 27 14:26:13.822146 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.822022 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n" event={"ID":"970c35db-69b6-414d-a853-6ad124b7f244","Type":"ContainerDied","Data":"06f0246a2bfecc599fe4bc78ea14615fbe219d03b6d25b68e8b6a9ecfa24cdbb"} Jul 27 14:26:13.822146 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.822039 2555 scope.go:117] "RemoveContainer" containerID="4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f" Jul 27 14:26:13.831047 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.831025 2555 scope.go:117] "RemoveContainer" containerID="efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557" Jul 27 14:26:13.839564 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.839542 2555 scope.go:117] "RemoveContainer" containerID="9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2" Jul 27 14:26:13.845455 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.845427 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n"] Jul 27 14:26:13.848714 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.848683 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-z5x6n"] Jul 27 14:26:13.849957 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.849936 2555 scope.go:117] "RemoveContainer" containerID="9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b" Jul 27 14:26:13.858558 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.858534 2555 scope.go:117] "RemoveContainer" containerID="5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f" Jul 27 14:26:13.866863 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.866841 2555 scope.go:117] "RemoveContainer" containerID="38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22" Jul 27 14:26:13.875216 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.875192 2555 scope.go:117] "RemoveContainer" containerID="4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f" Jul 27 14:26:13.875520 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:26:13.875500 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f\": container with ID starting with 4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f not found: ID does not exist" containerID="4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f" Jul 27 14:26:13.875606 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.875529 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f"} err="failed to get container status \"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f\": rpc error: code = NotFound desc = could not find container \"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f\": container with ID starting with 4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f not found: ID does not exist" Jul 27 14:26:13.875606 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.875551 2555 scope.go:117] "RemoveContainer" containerID="efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557" Jul 27 14:26:13.875781 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:26:13.875765 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557\": container with ID starting with efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557 not found: ID does not exist" containerID="efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557" Jul 27 14:26:13.875817 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.875785 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557"} err="failed to get container status \"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557\": rpc error: code = NotFound desc = could not find container \"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557\": container with ID starting with efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557 not found: ID does not exist" Jul 27 14:26:13.875817 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.875798 2555 scope.go:117] "RemoveContainer" containerID="9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2" Jul 27 14:26:13.876004 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:26:13.875990 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2\": container with ID starting with 9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2 not found: ID does not exist" containerID="9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2" Jul 27 14:26:13.876045 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876007 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2"} err="failed to get container status \"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2\": rpc error: code = NotFound desc = could not find container \"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2\": container with ID starting with 9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2 not found: ID does not exist" Jul 27 14:26:13.876045 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876020 2555 scope.go:117] "RemoveContainer" containerID="9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b" Jul 27 14:26:13.876218 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:26:13.876203 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b\": container with ID starting with 9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b not found: ID does not exist" containerID="9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b" Jul 27 14:26:13.876259 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876220 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b"} err="failed to get container status \"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b\": rpc error: code = NotFound desc = could not find container \"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b\": container with ID starting with 9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b not found: ID does not exist" Jul 27 14:26:13.876259 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876232 2555 scope.go:117] "RemoveContainer" containerID="5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f" Jul 27 14:26:13.876437 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:26:13.876417 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f\": container with ID starting with 5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f not found: ID does not exist" containerID="5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f" Jul 27 14:26:13.876474 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876440 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f"} err="failed to get container status \"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f\": rpc error: code = NotFound desc = could not find container \"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f\": container with ID starting with 5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f not found: ID does not exist" Jul 27 14:26:13.876474 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876453 2555 scope.go:117] "RemoveContainer" containerID="38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22" Jul 27 14:26:13.876680 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:26:13.876644 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22\": container with ID starting with 38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22 not found: ID does not exist" containerID="38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22" Jul 27 14:26:13.876722 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876685 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22"} err="failed to get container status \"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22\": rpc error: code = NotFound desc = could not find container \"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22\": container with ID starting with 38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22 not found: ID does not exist" Jul 27 14:26:13.876722 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876708 2555 scope.go:117] "RemoveContainer" containerID="4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f" Jul 27 14:26:13.876948 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876931 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f"} err="failed to get container status \"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f\": rpc error: code = NotFound desc = could not find container \"4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f\": container with ID starting with 4c013591a967bd8a50fbbf137d3e8737b59449a3c63914f628846b6363fc634f not found: ID does not exist" Jul 27 14:26:13.877001 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.876948 2555 scope.go:117] "RemoveContainer" containerID="efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557" Jul 27 14:26:13.877146 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877126 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557"} err="failed to get container status \"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557\": rpc error: code = NotFound desc = could not find container \"efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557\": container with ID starting with efa7433780c7844356ac5ffd19af4a7ba5930614e5954f1f42971fff52488557 not found: ID does not exist" Jul 27 14:26:13.877190 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877147 2555 scope.go:117] "RemoveContainer" containerID="9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2" Jul 27 14:26:13.877325 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877311 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2"} err="failed to get container status \"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2\": rpc error: code = NotFound desc = could not find container \"9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2\": container with ID starting with 9af9cac3f42c4a3a95304ad3e9c2dda6bffdb8944eafd0d2cee1c974c71564b2 not found: ID does not exist" Jul 27 14:26:13.877362 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877325 2555 scope.go:117] "RemoveContainer" containerID="9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b" Jul 27 14:26:13.877508 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877488 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b"} err="failed to get container status \"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b\": rpc error: code = NotFound desc = could not find container \"9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b\": container with ID starting with 9e51333944be0662270e2579632311443d80cf8b02cca69b9ebcb2698df3480b not found: ID does not exist" Jul 27 14:26:13.877554 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877509 2555 scope.go:117] "RemoveContainer" containerID="5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f" Jul 27 14:26:13.877746 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877729 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f"} err="failed to get container status \"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f\": rpc error: code = NotFound desc = could not find container \"5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f\": container with ID starting with 5d7fff8361c3674a69a4a033f3e0d46658fcaa3d9e52a5316e8c45ac26918e3f not found: ID does not exist" Jul 27 14:26:13.877791 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877747 2555 scope.go:117] "RemoveContainer" containerID="38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22" Jul 27 14:26:13.877971 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:13.877951 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22"} err="failed to get container status \"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22\": rpc error: code = NotFound desc = could not find container \"38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22\": container with ID starting with 38c4724c2a0178f601a22860c0231dd4c4869ea9e02028d1171c06ed781e9c22 not found: ID does not exist" Jul 27 14:26:15.148115 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:15.148080 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="970c35db-69b6-414d-a853-6ad124b7f244" path="/var/lib/kubelet/pods/970c35db-69b6-414d-a853-6ad124b7f244/volumes" Jul 27 14:26:57.026571 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:57.026541 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:26:57.030028 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:26:57.030007 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:27:08.935692 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.935643 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8"] Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.935998 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="feast-init" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936009 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="feast-init" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936025 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="offline" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936031 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="offline" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936036 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="ui" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936042 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="ui" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936053 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="registry" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936058 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="registry" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936067 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="online" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936072 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="online" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936081 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="feast-apply" Jul 27 14:27:08.936085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936086 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="feast-apply" Jul 27 14:27:08.936466 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936136 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="ui" Jul 27 14:27:08.936466 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936145 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="registry" Jul 27 14:27:08.936466 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936152 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="offline" Jul 27 14:27:08.936466 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.936159 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="970c35db-69b6-414d-a853-6ad124b7f244" containerName="online" Jul 27 14:27:08.941097 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.941075 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:08.943647 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.943618 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 27 14:27:08.943805 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.943696 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 27 14:27:08.943805 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.943780 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 27 14:27:08.943805 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.943780 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 27 14:27:08.943805 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.943796 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 27 14:27:08.944761 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.944740 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-px7d2\"" Jul 27 14:27:08.944866 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.944741 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Jul 27 14:27:08.944866 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.944840 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 27 14:27:08.950148 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:08.950122 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8"] Jul 27 14:27:09.036524 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.036487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4l8m\" (UniqueName: \"kubernetes.io/projected/330f9a65-b8c2-46e3-a842-d514543dde83-kube-api-access-w4l8m\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.036733 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.036555 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/330f9a65-b8c2-46e3-a842-d514543dde83-registry-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.036733 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.036584 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-offline-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.036733 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.036605 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/330f9a65-b8c2-46e3-a842-d514543dde83-feast-data\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.036733 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.036624 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-ui-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.036733 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.036726 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-online-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.137995 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.137955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4l8m\" (UniqueName: \"kubernetes.io/projected/330f9a65-b8c2-46e3-a842-d514543dde83-kube-api-access-w4l8m\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.138156 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.138021 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/330f9a65-b8c2-46e3-a842-d514543dde83-registry-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.138156 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.138047 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-offline-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.138156 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.138071 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/330f9a65-b8c2-46e3-a842-d514543dde83-feast-data\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.138156 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.138090 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-ui-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.138304 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.138227 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-online-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.138447 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.138430 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/330f9a65-b8c2-46e3-a842-d514543dde83-feast-data\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.138760 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.138737 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/330f9a65-b8c2-46e3-a842-d514543dde83-registry-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.140718 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.140687 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-online-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.140831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.140697 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-ui-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.140831 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.140730 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-offline-tls\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.145283 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.145258 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4l8m\" (UniqueName: \"kubernetes.io/projected/330f9a65-b8c2-46e3-a842-d514543dde83-kube-api-access-w4l8m\") pod \"feast-simple-feast-remote-setup-646d986757-xrrk8\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.251415 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.251339 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:09.385255 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:09.385230 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8"] Jul 27 14:27:09.387721 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:27:09.387689 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod330f9a65_b8c2_46e3_a842_d514543dde83.slice/crio-6872ceb6828b2120f856cc80affdccadb2a241da351c0b115450f7f544003558 WatchSource:0}: Error finding container 6872ceb6828b2120f856cc80affdccadb2a241da351c0b115450f7f544003558: Status 404 returned error can't find the container with id 6872ceb6828b2120f856cc80affdccadb2a241da351c0b115450f7f544003558 Jul 27 14:27:10.070720 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:10.070680 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerStarted","Data":"490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b"} Jul 27 14:27:10.071119 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:10.070740 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerStarted","Data":"6872ceb6828b2120f856cc80affdccadb2a241da351c0b115450f7f544003558"} Jul 27 14:27:15.094039 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:15.093956 2555 generic.go:356] "Generic (PLEG): container finished" podID="330f9a65-b8c2-46e3-a842-d514543dde83" containerID="490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b" exitCode=0 Jul 27 14:27:15.094388 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:15.094027 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerDied","Data":"490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b"} Jul 27 14:27:16.100838 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:16.100804 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerStarted","Data":"4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5"} Jul 27 14:27:22.133379 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:22.133346 2555 generic.go:356] "Generic (PLEG): container finished" podID="330f9a65-b8c2-46e3-a842-d514543dde83" containerID="4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5" exitCode=0 Jul 27 14:27:22.133766 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:22.133397 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerDied","Data":"4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5"} Jul 27 14:27:23.149324 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:23.149281 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerStarted","Data":"d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b"} Jul 27 14:27:23.149821 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:23.149332 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerStarted","Data":"d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d"} Jul 27 14:27:23.149821 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:23.149350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerStarted","Data":"9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651"} Jul 27 14:27:23.175006 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:23.174940 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podStartSLOduration=15.174915772 podStartE2EDuration="15.174915772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:27:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:27:23.172203702 +0000 UTC m=+626.539465751" watchObservedRunningTime="2026-07-27 14:27:23.174915772 +0000 UTC m=+626.542177822" Jul 27 14:27:24.251551 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:24.251504 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:24.251945 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:24.251572 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:24.251945 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:24.251594 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:24.253689 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:24.253635 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 27 14:27:24.253823 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:24.253697 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Jul 27 14:27:24.253823 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:24.253742 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 27 14:27:27.252090 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:27.252036 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Jul 27 14:27:27.252521 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:27.252036 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 27 14:27:27.252521 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:27.252053 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 27 14:27:29.252092 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:29.252058 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:29.252558 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:29.252128 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:29.252558 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:29.252163 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:30.252581 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:30.252552 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:30.252970 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:30.252699 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:30.258791 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:30.258762 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:31.186264 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:31.186236 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:31.186430 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:31.186293 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:27:31.190266 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:27:31.190242 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:28:32.308505 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.308466 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8"] Jul 27 14:28:32.309102 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.308773 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" containerID="cri-o://9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651" gracePeriod=30 Jul 27 14:28:32.309102 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.308898 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="offline" containerID="cri-o://d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d" gracePeriod=30 Jul 27 14:28:32.309102 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.309089 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" containerID="cri-o://d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b" gracePeriod=30 Jul 27 14:28:32.466889 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.466857 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp"] Jul 27 14:28:32.467113 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.467084 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="registry" containerID="cri-o://3c379c057c094effd9a6f35c802f8d8f300b72aedae10597dee3918647e2bab4" gracePeriod=30 Jul 27 14:28:32.467323 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.467143 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="offline" containerID="cri-o://a781da4660b30fd8863917b6ff416c761f6f62406d350e8f9f937097ac403bef" gracePeriod=30 Jul 27 14:28:32.467414 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.467145 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" containerID="cri-o://7e63082ff260211ae3af5eda4f43752bf52478c64258549498f258e876c939ca" gracePeriod=30 Jul 27 14:28:32.467472 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:32.467153 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" containerID="cri-o://849c6e77ca18f8c556582f7037ff359a962d185ec6e5df45022a53670a70b3c0" gracePeriod=30 Jul 27 14:28:33.458556 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:33.458524 2555 generic.go:356] "Generic (PLEG): container finished" podID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerID="849c6e77ca18f8c556582f7037ff359a962d185ec6e5df45022a53670a70b3c0" exitCode=0 Jul 27 14:28:33.458556 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:33.458547 2555 generic.go:356] "Generic (PLEG): container finished" podID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerID="7e63082ff260211ae3af5eda4f43752bf52478c64258549498f258e876c939ca" exitCode=0 Jul 27 14:28:33.459048 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:33.458603 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerDied","Data":"849c6e77ca18f8c556582f7037ff359a962d185ec6e5df45022a53670a70b3c0"} Jul 27 14:28:33.459048 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:33.458643 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerDied","Data":"7e63082ff260211ae3af5eda4f43752bf52478c64258549498f258e876c939ca"} Jul 27 14:28:33.461301 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:33.461277 2555 generic.go:356] "Generic (PLEG): container finished" podID="330f9a65-b8c2-46e3-a842-d514543dde83" containerID="d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b" exitCode=0 Jul 27 14:28:33.461301 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:33.461298 2555 generic.go:356] "Generic (PLEG): container finished" podID="330f9a65-b8c2-46e3-a842-d514543dde83" containerID="9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651" exitCode=0 Jul 27 14:28:33.461451 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:33.461347 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerDied","Data":"d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b"} Jul 27 14:28:33.461451 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:33.461383 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerDied","Data":"9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651"} Jul 27 14:28:35.785210 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:35.785162 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 27 14:28:35.785605 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:35.785178 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 27 14:28:41.186240 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:41.186192 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 27 14:28:41.186746 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:41.186202 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 27 14:28:45.785206 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:45.785164 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 27 14:28:45.785611 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:45.785174 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 27 14:28:51.186335 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:51.186234 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 27 14:28:51.186335 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:51.186297 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 27 14:28:55.785253 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:55.785204 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 27 14:28:55.785694 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:55.785228 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 27 14:28:55.785694 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:55.785337 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:28:55.785694 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:28:55.785496 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:29:01.186041 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:01.185992 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 27 14:29:01.186041 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:01.186021 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 27 14:29:01.186606 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:01.186132 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:29:01.186606 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:01.186179 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:29:02.495361 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.495333 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:29:02.556108 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.556078 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-offline-tls\") pod \"330f9a65-b8c2-46e3-a842-d514543dde83\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " Jul 27 14:29:02.556108 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.556108 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-online-tls\") pod \"330f9a65-b8c2-46e3-a842-d514543dde83\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " Jul 27 14:29:02.556291 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.556135 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-ui-tls\") pod \"330f9a65-b8c2-46e3-a842-d514543dde83\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " Jul 27 14:29:02.556291 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.556177 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/330f9a65-b8c2-46e3-a842-d514543dde83-feast-data\") pod \"330f9a65-b8c2-46e3-a842-d514543dde83\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " Jul 27 14:29:02.556291 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.556202 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/330f9a65-b8c2-46e3-a842-d514543dde83-registry-tls\") pod \"330f9a65-b8c2-46e3-a842-d514543dde83\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " Jul 27 14:29:02.556291 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.556230 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4l8m\" (UniqueName: \"kubernetes.io/projected/330f9a65-b8c2-46e3-a842-d514543dde83-kube-api-access-w4l8m\") pod \"330f9a65-b8c2-46e3-a842-d514543dde83\" (UID: \"330f9a65-b8c2-46e3-a842-d514543dde83\") " Jul 27 14:29:02.556570 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.556533 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/330f9a65-b8c2-46e3-a842-d514543dde83-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "330f9a65-b8c2-46e3-a842-d514543dde83" (UID: "330f9a65-b8c2-46e3-a842-d514543dde83"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 14:29:02.556749 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.556719 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/330f9a65-b8c2-46e3-a842-d514543dde83-feast-data" (OuterVolumeSpecName: "feast-data") pod "330f9a65-b8c2-46e3-a842-d514543dde83" (UID: "330f9a65-b8c2-46e3-a842-d514543dde83"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:29:02.558324 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.558290 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "330f9a65-b8c2-46e3-a842-d514543dde83" (UID: "330f9a65-b8c2-46e3-a842-d514543dde83"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:29:02.558324 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.558317 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-online-tls" (OuterVolumeSpecName: "online-tls") pod "330f9a65-b8c2-46e3-a842-d514543dde83" (UID: "330f9a65-b8c2-46e3-a842-d514543dde83"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:29:02.558444 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.558346 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/330f9a65-b8c2-46e3-a842-d514543dde83-kube-api-access-w4l8m" (OuterVolumeSpecName: "kube-api-access-w4l8m") pod "330f9a65-b8c2-46e3-a842-d514543dde83" (UID: "330f9a65-b8c2-46e3-a842-d514543dde83"). InnerVolumeSpecName "kube-api-access-w4l8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:29:02.558444 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.558372 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "330f9a65-b8c2-46e3-a842-d514543dde83" (UID: "330f9a65-b8c2-46e3-a842-d514543dde83"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:29:02.591094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.591023 2555 generic.go:356] "Generic (PLEG): container finished" podID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerID="a781da4660b30fd8863917b6ff416c761f6f62406d350e8f9f937097ac403bef" exitCode=137 Jul 27 14:29:02.591094 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.591052 2555 generic.go:356] "Generic (PLEG): container finished" podID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerID="3c379c057c094effd9a6f35c802f8d8f300b72aedae10597dee3918647e2bab4" exitCode=137 Jul 27 14:29:02.591242 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.591100 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerDied","Data":"a781da4660b30fd8863917b6ff416c761f6f62406d350e8f9f937097ac403bef"} Jul 27 14:29:02.591242 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.591145 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerDied","Data":"3c379c057c094effd9a6f35c802f8d8f300b72aedae10597dee3918647e2bab4"} Jul 27 14:29:02.593324 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.593301 2555 generic.go:356] "Generic (PLEG): container finished" podID="330f9a65-b8c2-46e3-a842-d514543dde83" containerID="d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d" exitCode=137 Jul 27 14:29:02.593422 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.593362 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" Jul 27 14:29:02.593422 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.593383 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerDied","Data":"d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d"} Jul 27 14:29:02.593506 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.593428 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8" event={"ID":"330f9a65-b8c2-46e3-a842-d514543dde83","Type":"ContainerDied","Data":"6872ceb6828b2120f856cc80affdccadb2a241da351c0b115450f7f544003558"} Jul 27 14:29:02.593506 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.593459 2555 scope.go:117] "RemoveContainer" containerID="d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b" Jul 27 14:29:02.603724 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.603614 2555 scope.go:117] "RemoveContainer" containerID="d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d" Jul 27 14:29:02.612129 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.612110 2555 scope.go:117] "RemoveContainer" containerID="9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651" Jul 27 14:29:02.616304 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.616283 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8"] Jul 27 14:29:02.619189 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.619170 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-646d986757-xrrk8"] Jul 27 14:29:02.620534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.620514 2555 scope.go:117] "RemoveContainer" containerID="4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5" Jul 27 14:29:02.627714 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.627699 2555 scope.go:117] "RemoveContainer" containerID="490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b" Jul 27 14:29:02.634594 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.634579 2555 scope.go:117] "RemoveContainer" containerID="d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b" Jul 27 14:29:02.634905 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:29:02.634886 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b\": container with ID starting with d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b not found: ID does not exist" containerID="d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b" Jul 27 14:29:02.634974 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.634911 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b"} err="failed to get container status \"d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b\": rpc error: code = NotFound desc = could not find container \"d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b\": container with ID starting with d4386607da3a3691b8a7c7f78144b86aadc3af2abd2ba426f8e8ed990bf3f07b not found: ID does not exist" Jul 27 14:29:02.634974 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.634928 2555 scope.go:117] "RemoveContainer" containerID="d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d" Jul 27 14:29:02.635166 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:29:02.635148 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d\": container with ID starting with d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d not found: ID does not exist" containerID="d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d" Jul 27 14:29:02.635203 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.635181 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d"} err="failed to get container status \"d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d\": rpc error: code = NotFound desc = could not find container \"d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d\": container with ID starting with d2baf799c3de80b181aa744b2e5bc1e1c762001695f2b946c6ded2e32a2ad48d not found: ID does not exist" Jul 27 14:29:02.635203 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.635197 2555 scope.go:117] "RemoveContainer" containerID="9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651" Jul 27 14:29:02.635411 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:29:02.635395 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651\": container with ID starting with 9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651 not found: ID does not exist" containerID="9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651" Jul 27 14:29:02.635462 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.635414 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651"} err="failed to get container status \"9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651\": rpc error: code = NotFound desc = could not find container \"9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651\": container with ID starting with 9ed4cf2c9446c7b496c5097d0ae13fef4c02f2a0cf926934ab647af498486651 not found: ID does not exist" Jul 27 14:29:02.635462 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.635427 2555 scope.go:117] "RemoveContainer" containerID="4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5" Jul 27 14:29:02.635630 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:29:02.635616 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5\": container with ID starting with 4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5 not found: ID does not exist" containerID="4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5" Jul 27 14:29:02.635695 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.635633 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5"} err="failed to get container status \"4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5\": rpc error: code = NotFound desc = could not find container \"4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5\": container with ID starting with 4599ce55d1a79305c751a8a989a14ff4b8d6467f5d6905367cb870949ca353e5 not found: ID does not exist" Jul 27 14:29:02.635695 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.635644 2555 scope.go:117] "RemoveContainer" containerID="490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b" Jul 27 14:29:02.635876 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:29:02.635859 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b\": container with ID starting with 490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b not found: ID does not exist" containerID="490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b" Jul 27 14:29:02.635921 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.635878 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b"} err="failed to get container status \"490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b\": rpc error: code = NotFound desc = could not find container \"490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b\": container with ID starting with 490017ca76172433ef339e777c65286e8ca1148a2b23c10a3f168e37a175373b not found: ID does not exist" Jul 27 14:29:02.657032 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.657011 2555 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/330f9a65-b8c2-46e3-a842-d514543dde83-feast-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:02.657032 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.657030 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/330f9a65-b8c2-46e3-a842-d514543dde83-registry-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:02.657139 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.657040 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4l8m\" (UniqueName: \"kubernetes.io/projected/330f9a65-b8c2-46e3-a842-d514543dde83-kube-api-access-w4l8m\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:02.657139 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.657050 2555 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-offline-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:02.657139 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.657058 2555 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-online-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:02.657139 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:02.657066 2555 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/330f9a65-b8c2-46e3-a842-d514543dde83-ui-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:03.102417 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.102391 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:29:03.148281 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.148253 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" path="/var/lib/kubelet/pods/330f9a65-b8c2-46e3-a842-d514543dde83/volumes" Jul 27 14:29:03.160342 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.160318 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls\") pod \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " Jul 27 14:29:03.160449 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.160361 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0645d16f-fc2f-4e72-af20-cd778dfdc28c-feast-data\") pod \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " Jul 27 14:29:03.160449 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.160391 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-offline-tls\") pod \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " Jul 27 14:29:03.160449 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.160418 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-registry-tls\") pod \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " Jul 27 14:29:03.160559 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.160473 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk4lr\" (UniqueName: \"kubernetes.io/projected/0645d16f-fc2f-4e72-af20-cd778dfdc28c-kube-api-access-qk4lr\") pod \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " Jul 27 14:29:03.160559 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.160542 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls\") pod \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\" (UID: \"0645d16f-fc2f-4e72-af20-cd778dfdc28c\") " Jul 27 14:29:03.161000 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.160976 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0645d16f-fc2f-4e72-af20-cd778dfdc28c-feast-data" (OuterVolumeSpecName: "feast-data") pod "0645d16f-fc2f-4e72-af20-cd778dfdc28c" (UID: "0645d16f-fc2f-4e72-af20-cd778dfdc28c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:29:03.162453 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.162435 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0645d16f-fc2f-4e72-af20-cd778dfdc28c" (UID: "0645d16f-fc2f-4e72-af20-cd778dfdc28c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:29:03.162964 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.162933 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0645d16f-fc2f-4e72-af20-cd778dfdc28c" (UID: "0645d16f-fc2f-4e72-af20-cd778dfdc28c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:29:03.162964 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.162953 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0645d16f-fc2f-4e72-af20-cd778dfdc28c" (UID: "0645d16f-fc2f-4e72-af20-cd778dfdc28c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:29:03.163068 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.162999 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0645d16f-fc2f-4e72-af20-cd778dfdc28c-kube-api-access-qk4lr" (OuterVolumeSpecName: "kube-api-access-qk4lr") pod "0645d16f-fc2f-4e72-af20-cd778dfdc28c" (UID: "0645d16f-fc2f-4e72-af20-cd778dfdc28c"). InnerVolumeSpecName "kube-api-access-qk4lr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:29:03.163068 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.163000 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls" (OuterVolumeSpecName: "online-tls") pod "0645d16f-fc2f-4e72-af20-cd778dfdc28c" (UID: "0645d16f-fc2f-4e72-af20-cd778dfdc28c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:29:03.262075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.261978 2555 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0645d16f-fc2f-4e72-af20-cd778dfdc28c-feast-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:03.262075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.262009 2555 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-offline-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:03.262075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.262022 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-registry-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:03.262075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.262034 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk4lr\" (UniqueName: \"kubernetes.io/projected/0645d16f-fc2f-4e72-af20-cd778dfdc28c-kube-api-access-qk4lr\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:03.262075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.262053 2555 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-online-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:03.262075 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.262061 2555 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0645d16f-fc2f-4e72-af20-cd778dfdc28c-ui-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:29:03.600566 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.600475 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" event={"ID":"0645d16f-fc2f-4e72-af20-cd778dfdc28c","Type":"ContainerDied","Data":"42c555b39b9b09b5949f522ba4a18cbed0d663a8ccde931561efed2af349364a"} Jul 27 14:29:03.600566 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.600505 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp" Jul 27 14:29:03.600566 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.600537 2555 scope.go:117] "RemoveContainer" containerID="849c6e77ca18f8c556582f7037ff359a962d185ec6e5df45022a53670a70b3c0" Jul 27 14:29:03.609765 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.609743 2555 scope.go:117] "RemoveContainer" containerID="a781da4660b30fd8863917b6ff416c761f6f62406d350e8f9f937097ac403bef" Jul 27 14:29:03.617260 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.617241 2555 scope.go:117] "RemoveContainer" containerID="7e63082ff260211ae3af5eda4f43752bf52478c64258549498f258e876c939ca" Jul 27 14:29:03.622300 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.622276 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp"] Jul 27 14:29:03.625060 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.625041 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5479df6db8-pt7sp"] Jul 27 14:29:03.626213 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.626200 2555 scope.go:117] "RemoveContainer" containerID="3c379c057c094effd9a6f35c802f8d8f300b72aedae10597dee3918647e2bab4" Jul 27 14:29:03.633956 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.633935 2555 scope.go:117] "RemoveContainer" containerID="7382f07c44dfb143a825d6f3dd39113a0872e782df129b1548c07844c28119c8" Jul 27 14:29:03.641627 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:03.641611 2555 scope.go:117] "RemoveContainer" containerID="9aa067e76d7cf7a0cb7aa6f115aead33046cd6ccf8ff1590b2579173ff6f11e4" Jul 27 14:29:05.147241 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:05.147210 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" path="/var/lib/kubelet/pods/0645d16f-fc2f-4e72-af20-cd778dfdc28c/volumes" Jul 27 14:29:15.270175 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270140 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-vjrwv"] Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270459 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="offline" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270470 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="offline" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270480 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="feast-apply" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270486 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="feast-apply" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270493 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270498 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270507 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270513 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270529 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="feast-init" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270534 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="feast-init" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270541 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="registry" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270546 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="registry" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270551 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="feast-init" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270556 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="feast-init" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270564 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270568 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270574 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270580 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270587 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="feast-apply" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270592 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="feast-apply" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270598 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="offline" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270611 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="offline" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270678 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="online" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270698 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="online" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270704 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="offline" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270711 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="offline" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270718 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="330f9a65-b8c2-46e3-a842-d514543dde83" containerName="ui" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270724 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="ui" Jul 27 14:29:15.270822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.270731 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0645d16f-fc2f-4e72-af20-cd778dfdc28c" containerName="registry" Jul 27 14:29:15.275634 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.275614 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:29:15.278196 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.278173 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 27 14:29:15.278437 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.278417 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Jul 27 14:29:15.278538 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.278435 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-kch8h\"" Jul 27 14:29:15.279504 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.279482 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 27 14:29:15.281645 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.281624 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-vjrwv"] Jul 27 14:29:15.300241 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.300207 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-9k5cx"] Jul 27 14:29:15.303971 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.303952 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-9k5cx" Jul 27 14:29:15.309283 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.309255 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9k5cx"] Jul 27 14:29:15.378745 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.378708 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ccb16470-0240-46e3-96ae-41eba31b18e1-postgresdata\") pod \"postgres-864846d4f5-vjrwv\" (UID: \"ccb16470-0240-46e3-96ae-41eba31b18e1\") " pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:29:15.378936 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.378855 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bvt8\" (UniqueName: \"kubernetes.io/projected/ccb16470-0240-46e3-96ae-41eba31b18e1-kube-api-access-4bvt8\") pod \"postgres-864846d4f5-vjrwv\" (UID: \"ccb16470-0240-46e3-96ae-41eba31b18e1\") " pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:29:15.479854 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.479818 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bvt8\" (UniqueName: \"kubernetes.io/projected/ccb16470-0240-46e3-96ae-41eba31b18e1-kube-api-access-4bvt8\") pod \"postgres-864846d4f5-vjrwv\" (UID: \"ccb16470-0240-46e3-96ae-41eba31b18e1\") " pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:29:15.480019 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.479868 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ccb16470-0240-46e3-96ae-41eba31b18e1-postgresdata\") pod \"postgres-864846d4f5-vjrwv\" (UID: \"ccb16470-0240-46e3-96ae-41eba31b18e1\") " pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:29:15.480019 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.479932 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqnk4\" (UniqueName: \"kubernetes.io/projected/bd7e955a-bd23-4aa8-9af1-9e2fff2f6357-kube-api-access-pqnk4\") pod \"redis-69bbd798b6-9k5cx\" (UID: \"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357\") " pod="test-ns-feast/redis-69bbd798b6-9k5cx" Jul 27 14:29:15.480273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.480254 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ccb16470-0240-46e3-96ae-41eba31b18e1-postgresdata\") pod \"postgres-864846d4f5-vjrwv\" (UID: \"ccb16470-0240-46e3-96ae-41eba31b18e1\") " pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:29:15.488025 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.487999 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bvt8\" (UniqueName: \"kubernetes.io/projected/ccb16470-0240-46e3-96ae-41eba31b18e1-kube-api-access-4bvt8\") pod \"postgres-864846d4f5-vjrwv\" (UID: \"ccb16470-0240-46e3-96ae-41eba31b18e1\") " pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:29:15.581304 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.581206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqnk4\" (UniqueName: \"kubernetes.io/projected/bd7e955a-bd23-4aa8-9af1-9e2fff2f6357-kube-api-access-pqnk4\") pod \"redis-69bbd798b6-9k5cx\" (UID: \"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357\") " pod="test-ns-feast/redis-69bbd798b6-9k5cx" Jul 27 14:29:15.589165 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.589139 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqnk4\" (UniqueName: \"kubernetes.io/projected/bd7e955a-bd23-4aa8-9af1-9e2fff2f6357-kube-api-access-pqnk4\") pod \"redis-69bbd798b6-9k5cx\" (UID: \"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357\") " pod="test-ns-feast/redis-69bbd798b6-9k5cx" Jul 27 14:29:15.595047 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.595024 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:29:15.615539 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.615517 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-9k5cx" Jul 27 14:29:15.739233 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.739203 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-vjrwv"] Jul 27 14:29:15.740930 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:29:15.740908 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccb16470_0240_46e3_96ae_41eba31b18e1.slice/crio-6d78b2cc8a24f375600511afa8d295e3ca26238a1cbc3a0960bfc7aa21cd56f8 WatchSource:0}: Error finding container 6d78b2cc8a24f375600511afa8d295e3ca26238a1cbc3a0960bfc7aa21cd56f8: Status 404 returned error can't find the container with id 6d78b2cc8a24f375600511afa8d295e3ca26238a1cbc3a0960bfc7aa21cd56f8 Jul 27 14:29:15.748444 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.748425 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 14:29:15.752832 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:15.752728 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9k5cx"] Jul 27 14:29:15.755758 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:29:15.755727 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd7e955a_bd23_4aa8_9af1_9e2fff2f6357.slice/crio-78b38664c401b8648adc1c6f084c17ceac6132e72372e3b661d346850fb00593 WatchSource:0}: Error finding container 78b38664c401b8648adc1c6f084c17ceac6132e72372e3b661d346850fb00593: Status 404 returned error can't find the container with id 78b38664c401b8648adc1c6f084c17ceac6132e72372e3b661d346850fb00593 Jul 27 14:29:16.661061 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:16.661025 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-9k5cx" event={"ID":"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357","Type":"ContainerStarted","Data":"78b38664c401b8648adc1c6f084c17ceac6132e72372e3b661d346850fb00593"} Jul 27 14:29:16.662443 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:16.662414 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-vjrwv" event={"ID":"ccb16470-0240-46e3-96ae-41eba31b18e1","Type":"ContainerStarted","Data":"6d78b2cc8a24f375600511afa8d295e3ca26238a1cbc3a0960bfc7aa21cd56f8"} Jul 27 14:29:17.669582 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:17.669544 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-9k5cx" event={"ID":"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357","Type":"ContainerStarted","Data":"abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915"} Jul 27 14:29:17.687090 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:17.687023 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-9k5cx" podStartSLOduration=0.885165378 podStartE2EDuration="2.687003296s" totalImagesPullingTime="1.801837918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:29:17.685027538 +0000 UTC m=+741.052289583" watchObservedRunningTime="2026-07-27 14:29:17.687003296 +0000 UTC m=+741.054265321" Jul 27 14:29:21.690455 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:21.690416 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-vjrwv" event={"ID":"ccb16470-0240-46e3-96ae-41eba31b18e1","Type":"ContainerStarted","Data":"2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3"} Jul 27 14:29:21.705787 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:21.705730 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-vjrwv" podStartSLOduration=1.5560898829999998 podStartE2EDuration="6.705711994s" totalImagesPullingTime="5.149622111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:29:21.705646796 +0000 UTC m=+745.072908845" watchObservedRunningTime="2026-07-27 14:29:21.705711994 +0000 UTC m=+745.072974044" Jul 27 14:29:26.022241 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.022205 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7"] Jul 27 14:29:26.027399 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.027378 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.030155 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.030128 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 27 14:29:26.030273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.030191 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-8c46d\"" Jul 27 14:29:26.030273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.030132 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Jul 27 14:29:26.030273 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.030217 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Jul 27 14:29:26.030487 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.030474 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Jul 27 14:29:26.040299 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.040274 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7"] Jul 27 14:29:26.176856 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.176816 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5stf\" (UniqueName: \"kubernetes.io/projected/72269f8c-bb5a-4caf-90c3-32c92304de84-kube-api-access-p5stf\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.176856 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.176863 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72269f8c-bb5a-4caf-90c3-32c92304de84-feast-data\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.177067 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.176884 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-online-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.177067 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.176945 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-offline-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.177067 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.176985 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.278198 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.278116 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.278198 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.278181 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5stf\" (UniqueName: \"kubernetes.io/projected/72269f8c-bb5a-4caf-90c3-32c92304de84-kube-api-access-p5stf\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.278387 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.278209 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72269f8c-bb5a-4caf-90c3-32c92304de84-feast-data\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.278387 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.278228 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-online-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.278387 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:29:26.278268 2555 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Jul 27 14:29:26.278387 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.278274 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-offline-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.278387 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:29:26.278357 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls podName:72269f8c-bb5a-4caf-90c3-32c92304de84 nodeName:}" failed. No retries permitted until 2026-07-27 14:29:26.778337656 +0000 UTC m=+750.145599696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls") pod "feast-credit-scoring-7886f895c6-ghjm7" (UID: "72269f8c-bb5a-4caf-90c3-32c92304de84") : secret "feast-credit-scoring-registry-tls" not found Jul 27 14:29:26.278634 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.278617 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72269f8c-bb5a-4caf-90c3-32c92304de84-feast-data\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.280704 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.280679 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-offline-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.280807 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.280716 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-online-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.286490 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.286468 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5stf\" (UniqueName: \"kubernetes.io/projected/72269f8c-bb5a-4caf-90c3-32c92304de84-kube-api-access-p5stf\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.783857 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.783815 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.786259 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.786237 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls\") pod \"feast-credit-scoring-7886f895c6-ghjm7\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:26.936916 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:26.936882 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:27.070533 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:27.070505 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7"] Jul 27 14:29:27.071409 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:29:27.071382 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72269f8c_bb5a_4caf_90c3_32c92304de84.slice/crio-d70f6161ec029d331479bbf6e9fd76c0b093791b5c32d030242c5bbce632aa87 WatchSource:0}: Error finding container d70f6161ec029d331479bbf6e9fd76c0b093791b5c32d030242c5bbce632aa87: Status 404 returned error can't find the container with id d70f6161ec029d331479bbf6e9fd76c0b093791b5c32d030242c5bbce632aa87 Jul 27 14:29:27.717269 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:27.717227 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerStarted","Data":"939ae44881da8493d6e42bbc1f829b421241bdc2990f917fbabefde0015dd7d9"} Jul 27 14:29:27.717489 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:27.717315 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerStarted","Data":"d70f6161ec029d331479bbf6e9fd76c0b093791b5c32d030242c5bbce632aa87"} Jul 27 14:29:28.723652 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:28.723615 2555 generic.go:356] "Generic (PLEG): container finished" podID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerID="939ae44881da8493d6e42bbc1f829b421241bdc2990f917fbabefde0015dd7d9" exitCode=0 Jul 27 14:29:28.724109 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:28.723672 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerDied","Data":"939ae44881da8493d6e42bbc1f829b421241bdc2990f917fbabefde0015dd7d9"} Jul 27 14:29:29.729964 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:29.729927 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerStarted","Data":"9cdf4e830b336f534faf1250e87463a7a2d79b6fed4683307730bedcd31123f1"} Jul 27 14:29:35.760023 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:35.759988 2555 generic.go:356] "Generic (PLEG): container finished" podID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerID="9cdf4e830b336f534faf1250e87463a7a2d79b6fed4683307730bedcd31123f1" exitCode=0 Jul 27 14:29:35.760404 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:35.760044 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerDied","Data":"9cdf4e830b336f534faf1250e87463a7a2d79b6fed4683307730bedcd31123f1"} Jul 27 14:29:36.768889 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:36.768853 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerStarted","Data":"0510744f0fa76693b060c457681d541849aba35d60f751db263aed686bc7a2b3"} Jul 27 14:29:36.769378 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:36.768914 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerStarted","Data":"660dbb7dde6c17ce390b43384f966e80eebc04b3c1c8412e2db5d55609ea4723"} Jul 27 14:29:36.769378 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:36.768931 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerStarted","Data":"4d7dba96d5ca602de4227ddbec9697735999cbc1ccb4970b7c1513971a49dfde"} Jul 27 14:29:36.794744 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:36.794684 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podStartSLOduration=10.794637609 podStartE2EDuration="10.794637609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:29:36.793594724 +0000 UTC m=+760.160856776" watchObservedRunningTime="2026-07-27 14:29:36.794637609 +0000 UTC m=+760.161899660" Jul 27 14:29:36.937264 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:36.937225 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:36.937447 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:36.937304 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:36.937447 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:36.937329 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:38.938014 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:38.937977 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:38.938617 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:38.938593 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:38.938771 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:38.938631 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:38.939971 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:38.939936 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Jul 27 14:29:38.940080 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:38.940024 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Jul 27 14:29:38.940080 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:38.940027 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 27 14:29:41.938392 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:41.937906 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Jul 27 14:29:41.938930 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:41.938533 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:41.939495 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:41.939469 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:42.054145 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:42.054107 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:42.805762 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:42.805729 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:44.937495 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:44.937437 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Jul 27 14:29:47.938580 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:47.938545 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:29:47.939478 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:29:47.939458 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:30:55.623665 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.623615 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7"] Jul 27 14:30:55.624140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.623889 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="registry" containerID="cri-o://4d7dba96d5ca602de4227ddbec9697735999cbc1ccb4970b7c1513971a49dfde" gracePeriod=30 Jul 27 14:30:55.624140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.623923 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="offline" containerID="cri-o://0510744f0fa76693b060c457681d541849aba35d60f751db263aed686bc7a2b3" gracePeriod=30 Jul 27 14:30:55.624140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.623954 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="online" containerID="cri-o://660dbb7dde6c17ce390b43384f966e80eebc04b3c1c8412e2db5d55609ea4723" gracePeriod=30 Jul 27 14:30:55.642999 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.642960 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-vjrwv"] Jul 27 14:30:55.643163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.643139 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-vjrwv" podUID="ccb16470-0240-46e3-96ae-41eba31b18e1" containerName="postgres" containerID="cri-o://2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3" gracePeriod=30 Jul 27 14:30:55.651074 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.651028 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9k5cx"] Jul 27 14:30:55.651266 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.651239 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-9k5cx" podUID="bd7e955a-bd23-4aa8-9af1-9e2fff2f6357" containerName="redis" containerID="cri-o://abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915" gracePeriod=30 Jul 27 14:30:55.929426 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.929398 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-9k5cx" Jul 27 14:30:55.932858 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:55.932831 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:30:56.093939 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.093893 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ccb16470-0240-46e3-96ae-41eba31b18e1-postgresdata\") pod \"ccb16470-0240-46e3-96ae-41eba31b18e1\" (UID: \"ccb16470-0240-46e3-96ae-41eba31b18e1\") " Jul 27 14:30:56.094140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.093954 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bvt8\" (UniqueName: \"kubernetes.io/projected/ccb16470-0240-46e3-96ae-41eba31b18e1-kube-api-access-4bvt8\") pod \"ccb16470-0240-46e3-96ae-41eba31b18e1\" (UID: \"ccb16470-0240-46e3-96ae-41eba31b18e1\") " Jul 27 14:30:56.094140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.094066 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqnk4\" (UniqueName: \"kubernetes.io/projected/bd7e955a-bd23-4aa8-9af1-9e2fff2f6357-kube-api-access-pqnk4\") pod \"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357\" (UID: \"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357\") " Jul 27 14:30:56.096179 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.096139 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccb16470-0240-46e3-96ae-41eba31b18e1-kube-api-access-4bvt8" (OuterVolumeSpecName: "kube-api-access-4bvt8") pod "ccb16470-0240-46e3-96ae-41eba31b18e1" (UID: "ccb16470-0240-46e3-96ae-41eba31b18e1"). InnerVolumeSpecName "kube-api-access-4bvt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:30:56.096298 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.096216 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd7e955a-bd23-4aa8-9af1-9e2fff2f6357-kube-api-access-pqnk4" (OuterVolumeSpecName: "kube-api-access-pqnk4") pod "bd7e955a-bd23-4aa8-9af1-9e2fff2f6357" (UID: "bd7e955a-bd23-4aa8-9af1-9e2fff2f6357"). InnerVolumeSpecName "kube-api-access-pqnk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:30:56.122910 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.122870 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb16470-0240-46e3-96ae-41eba31b18e1-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "ccb16470-0240-46e3-96ae-41eba31b18e1" (UID: "ccb16470-0240-46e3-96ae-41eba31b18e1"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:30:56.142005 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.141971 2555 generic.go:356] "Generic (PLEG): container finished" podID="bd7e955a-bd23-4aa8-9af1-9e2fff2f6357" containerID="abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915" exitCode=0 Jul 27 14:30:56.142184 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.142044 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-9k5cx" Jul 27 14:30:56.142184 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.142045 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-9k5cx" event={"ID":"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357","Type":"ContainerDied","Data":"abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915"} Jul 27 14:30:56.142184 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.142169 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-9k5cx" event={"ID":"bd7e955a-bd23-4aa8-9af1-9e2fff2f6357","Type":"ContainerDied","Data":"78b38664c401b8648adc1c6f084c17ceac6132e72372e3b661d346850fb00593"} Jul 27 14:30:56.142355 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.142201 2555 scope.go:117] "RemoveContainer" containerID="abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915" Jul 27 14:30:56.143443 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.143423 2555 generic.go:356] "Generic (PLEG): container finished" podID="ccb16470-0240-46e3-96ae-41eba31b18e1" containerID="2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3" exitCode=0 Jul 27 14:30:56.143526 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.143503 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-vjrwv" event={"ID":"ccb16470-0240-46e3-96ae-41eba31b18e1","Type":"ContainerDied","Data":"2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3"} Jul 27 14:30:56.143526 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.143510 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-vjrwv" Jul 27 14:30:56.143594 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.143542 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-vjrwv" event={"ID":"ccb16470-0240-46e3-96ae-41eba31b18e1","Type":"ContainerDied","Data":"6d78b2cc8a24f375600511afa8d295e3ca26238a1cbc3a0960bfc7aa21cd56f8"} Jul 27 14:30:56.151736 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.151709 2555 scope.go:117] "RemoveContainer" containerID="abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915" Jul 27 14:30:56.152040 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:30:56.152021 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915\": container with ID starting with abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915 not found: ID does not exist" containerID="abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915" Jul 27 14:30:56.152121 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.152048 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915"} err="failed to get container status \"abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915\": rpc error: code = NotFound desc = could not find container \"abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915\": container with ID starting with abdeebb2622ff56ee594531d736f5f6de7d9198c1fb21e690aa1327a9193b915 not found: ID does not exist" Jul 27 14:30:56.152121 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.152065 2555 scope.go:117] "RemoveContainer" containerID="2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3" Jul 27 14:30:56.160732 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.160709 2555 scope.go:117] "RemoveContainer" containerID="2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3" Jul 27 14:30:56.161053 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:30:56.161028 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3\": container with ID starting with 2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3 not found: ID does not exist" containerID="2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3" Jul 27 14:30:56.161148 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.161062 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3"} err="failed to get container status \"2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3\": rpc error: code = NotFound desc = could not find container \"2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3\": container with ID starting with 2390baa10611133052b0537ab81f48fdd55e8bb3bbea0e690d626e19179dedb3 not found: ID does not exist" Jul 27 14:30:56.167966 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.167939 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9k5cx"] Jul 27 14:30:56.171105 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.171083 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9k5cx"] Jul 27 14:30:56.181949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.181921 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-vjrwv"] Jul 27 14:30:56.188837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.188809 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-vjrwv"] Jul 27 14:30:56.195297 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.195268 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqnk4\" (UniqueName: \"kubernetes.io/projected/bd7e955a-bd23-4aa8-9af1-9e2fff2f6357-kube-api-access-pqnk4\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:30:56.195406 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.195298 2555 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ccb16470-0240-46e3-96ae-41eba31b18e1-postgresdata\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:30:56.195406 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:56.195315 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bvt8\" (UniqueName: \"kubernetes.io/projected/ccb16470-0240-46e3-96ae-41eba31b18e1-kube-api-access-4bvt8\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:30:57.149485 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:57.149451 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd7e955a-bd23-4aa8-9af1-9e2fff2f6357" path="/var/lib/kubelet/pods/bd7e955a-bd23-4aa8-9af1-9e2fff2f6357/volumes" Jul 27 14:30:57.149917 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:57.149893 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccb16470-0240-46e3-96ae-41eba31b18e1" path="/var/lib/kubelet/pods/ccb16470-0240-46e3-96ae-41eba31b18e1/volumes" Jul 27 14:30:57.150885 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:57.150864 2555 generic.go:356] "Generic (PLEG): container finished" podID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerID="660dbb7dde6c17ce390b43384f966e80eebc04b3c1c8412e2db5d55609ea4723" exitCode=0 Jul 27 14:30:57.150979 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:30:57.150927 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerDied","Data":"660dbb7dde6c17ce390b43384f966e80eebc04b3c1c8412e2db5d55609ea4723"} Jul 27 14:31:02.803554 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:02.803512 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 27 14:31:12.803705 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:12.803632 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 27 14:31:22.803396 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:22.803349 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 27 14:31:22.803854 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:22.803503 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:31:26.274050 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.274022 2555 generic.go:356] "Generic (PLEG): container finished" podID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerID="0510744f0fa76693b060c457681d541849aba35d60f751db263aed686bc7a2b3" exitCode=137 Jul 27 14:31:26.274050 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.274046 2555 generic.go:356] "Generic (PLEG): container finished" podID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerID="4d7dba96d5ca602de4227ddbec9697735999cbc1ccb4970b7c1513971a49dfde" exitCode=137 Jul 27 14:31:26.274398 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.274066 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerDied","Data":"0510744f0fa76693b060c457681d541849aba35d60f751db263aed686bc7a2b3"} Jul 27 14:31:26.274398 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.274105 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerDied","Data":"4d7dba96d5ca602de4227ddbec9697735999cbc1ccb4970b7c1513971a49dfde"} Jul 27 14:31:26.274398 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.274123 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" event={"ID":"72269f8c-bb5a-4caf-90c3-32c92304de84","Type":"ContainerDied","Data":"d70f6161ec029d331479bbf6e9fd76c0b093791b5c32d030242c5bbce632aa87"} Jul 27 14:31:26.274398 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.274139 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d70f6161ec029d331479bbf6e9fd76c0b093791b5c32d030242c5bbce632aa87" Jul 27 14:31:26.276171 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.276154 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:31:26.334882 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.334851 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72269f8c-bb5a-4caf-90c3-32c92304de84-feast-data\") pod \"72269f8c-bb5a-4caf-90c3-32c92304de84\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " Jul 27 14:31:26.335051 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.334896 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5stf\" (UniqueName: \"kubernetes.io/projected/72269f8c-bb5a-4caf-90c3-32c92304de84-kube-api-access-p5stf\") pod \"72269f8c-bb5a-4caf-90c3-32c92304de84\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " Jul 27 14:31:26.335051 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.334926 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-online-tls\") pod \"72269f8c-bb5a-4caf-90c3-32c92304de84\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " Jul 27 14:31:26.335051 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.334960 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-offline-tls\") pod \"72269f8c-bb5a-4caf-90c3-32c92304de84\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " Jul 27 14:31:26.335051 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.334974 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls\") pod \"72269f8c-bb5a-4caf-90c3-32c92304de84\" (UID: \"72269f8c-bb5a-4caf-90c3-32c92304de84\") " Jul 27 14:31:26.337089 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.337052 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "72269f8c-bb5a-4caf-90c3-32c92304de84" (UID: "72269f8c-bb5a-4caf-90c3-32c92304de84"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:31:26.337200 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.337065 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "72269f8c-bb5a-4caf-90c3-32c92304de84" (UID: "72269f8c-bb5a-4caf-90c3-32c92304de84"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:31:26.337200 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.337094 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-online-tls" (OuterVolumeSpecName: "online-tls") pod "72269f8c-bb5a-4caf-90c3-32c92304de84" (UID: "72269f8c-bb5a-4caf-90c3-32c92304de84"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:31:26.337300 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.337280 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72269f8c-bb5a-4caf-90c3-32c92304de84-kube-api-access-p5stf" (OuterVolumeSpecName: "kube-api-access-p5stf") pod "72269f8c-bb5a-4caf-90c3-32c92304de84" (UID: "72269f8c-bb5a-4caf-90c3-32c92304de84"). InnerVolumeSpecName "kube-api-access-p5stf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:31:26.343156 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.343135 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72269f8c-bb5a-4caf-90c3-32c92304de84-feast-data" (OuterVolumeSpecName: "feast-data") pod "72269f8c-bb5a-4caf-90c3-32c92304de84" (UID: "72269f8c-bb5a-4caf-90c3-32c92304de84"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:31:26.435748 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.435724 2555 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72269f8c-bb5a-4caf-90c3-32c92304de84-feast-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:31:26.435748 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.435747 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5stf\" (UniqueName: \"kubernetes.io/projected/72269f8c-bb5a-4caf-90c3-32c92304de84-kube-api-access-p5stf\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:31:26.435911 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.435760 2555 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-online-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:31:26.435911 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.435768 2555 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-offline-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:31:26.435911 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:26.435775 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72269f8c-bb5a-4caf-90c3-32c92304de84-registry-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:31:27.278140 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:27.278105 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7" Jul 27 14:31:27.296875 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:27.296844 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7"] Jul 27 14:31:27.302709 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:27.302686 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-7886f895c6-ghjm7"] Jul 27 14:31:29.147177 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:29.147140 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" path="/var/lib/kubelet/pods/72269f8c-bb5a-4caf-90c3-32c92304de84/volumes" Jul 27 14:31:57.062394 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:57.062303 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:31:57.067651 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:31:57.067629 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:33:25.580534 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580452 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6tqvl"] Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580803 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="feast-init" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580815 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="feast-init" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580828 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="registry" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580834 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="registry" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580844 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="online" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580849 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="online" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580857 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="feast-apply" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580863 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="feast-apply" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580872 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccb16470-0240-46e3-96ae-41eba31b18e1" containerName="postgres" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580876 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccb16470-0240-46e3-96ae-41eba31b18e1" containerName="postgres" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580887 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd7e955a-bd23-4aa8-9af1-9e2fff2f6357" containerName="redis" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580892 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd7e955a-bd23-4aa8-9af1-9e2fff2f6357" containerName="redis" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580897 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="offline" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580902 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="offline" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580969 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd7e955a-bd23-4aa8-9af1-9e2fff2f6357" containerName="redis" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580978 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccb16470-0240-46e3-96ae-41eba31b18e1" containerName="postgres" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580984 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="online" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580989 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="offline" Jul 27 14:33:25.582949 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.580995 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="72269f8c-bb5a-4caf-90c3-32c92304de84" containerName="registry" Jul 27 14:33:25.583948 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.583818 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" Jul 27 14:33:25.586543 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.586516 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Jul 27 14:33:25.587600 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.587582 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Jul 27 14:33:25.587729 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.587688 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-9f25j\"" Jul 27 14:33:25.594456 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.594434 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6tqvl"] Jul 27 14:33:25.642511 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.642477 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgmpt\" (UniqueName: \"kubernetes.io/projected/21071ca9-242f-453d-9572-77f744c97d1f-kube-api-access-fgmpt\") pod \"redis-69bbd798b6-6tqvl\" (UID: \"21071ca9-242f-453d-9572-77f744c97d1f\") " pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" Jul 27 14:33:25.743591 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.743551 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgmpt\" (UniqueName: \"kubernetes.io/projected/21071ca9-242f-453d-9572-77f744c97d1f-kube-api-access-fgmpt\") pod \"redis-69bbd798b6-6tqvl\" (UID: \"21071ca9-242f-453d-9572-77f744c97d1f\") " pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" Jul 27 14:33:25.752369 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.752337 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgmpt\" (UniqueName: \"kubernetes.io/projected/21071ca9-242f-453d-9572-77f744c97d1f-kube-api-access-fgmpt\") pod \"redis-69bbd798b6-6tqvl\" (UID: \"21071ca9-242f-453d-9572-77f744c97d1f\") " pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" Jul 27 14:33:25.900341 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:25.900256 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" Jul 27 14:33:26.030614 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:26.030582 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6tqvl"] Jul 27 14:33:26.033802 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:33:26.033771 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21071ca9_242f_453d_9572_77f744c97d1f.slice/crio-c6eec998ad52c767d1bc03b89a9d443f68226f831882dde86f029471cc551f23 WatchSource:0}: Error finding container c6eec998ad52c767d1bc03b89a9d443f68226f831882dde86f029471cc551f23: Status 404 returned error can't find the container with id c6eec998ad52c767d1bc03b89a9d443f68226f831882dde86f029471cc551f23 Jul 27 14:33:26.782635 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:26.782603 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" event={"ID":"21071ca9-242f-453d-9572-77f744c97d1f","Type":"ContainerStarted","Data":"bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2"} Jul 27 14:33:26.783029 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:26.782643 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" event={"ID":"21071ca9-242f-453d-9572-77f744c97d1f","Type":"ContainerStarted","Data":"c6eec998ad52c767d1bc03b89a9d443f68226f831882dde86f029471cc551f23"} Jul 27 14:33:26.797514 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:26.797448 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" podStartSLOduration=1.79742639 podStartE2EDuration="1.79742639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:33:26.796401848 +0000 UTC m=+990.163663897" watchObservedRunningTime="2026-07-27 14:33:26.79742639 +0000 UTC m=+990.164688438" Jul 27 14:33:31.025085 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.025053 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-85x5t"] Jul 27 14:33:31.028767 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.028745 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:33:31.031549 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.031528 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Jul 27 14:33:31.034855 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.034831 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-85x5t"] Jul 27 14:33:31.091574 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.091539 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlmzs\" (UniqueName: \"kubernetes.io/projected/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-kube-api-access-rlmzs\") pod \"postgres-864846d4f5-85x5t\" (UID: \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\") " pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:33:31.091791 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.091583 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-postgresdata\") pod \"postgres-864846d4f5-85x5t\" (UID: \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\") " pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:33:31.192879 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.192844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-postgresdata\") pod \"postgres-864846d4f5-85x5t\" (UID: \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\") " pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:33:31.193031 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.192935 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlmzs\" (UniqueName: \"kubernetes.io/projected/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-kube-api-access-rlmzs\") pod \"postgres-864846d4f5-85x5t\" (UID: \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\") " pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:33:31.193232 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.193198 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-postgresdata\") pod \"postgres-864846d4f5-85x5t\" (UID: \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\") " pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:33:31.201255 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.201228 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlmzs\" (UniqueName: \"kubernetes.io/projected/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-kube-api-access-rlmzs\") pod \"postgres-864846d4f5-85x5t\" (UID: \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\") " pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:33:31.340223 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.340128 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:33:31.466640 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.466613 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-85x5t"] Jul 27 14:33:31.471764 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:33:31.471729 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc65dd0d_87c8_4a37_bc34_b6b7dcaf98a7.slice/crio-d5b8fca462b40320a78488abc04e9972ab68813828cedf9bb027a25e02941efb WatchSource:0}: Error finding container d5b8fca462b40320a78488abc04e9972ab68813828cedf9bb027a25e02941efb: Status 404 returned error can't find the container with id d5b8fca462b40320a78488abc04e9972ab68813828cedf9bb027a25e02941efb Jul 27 14:33:31.807078 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.807048 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" event={"ID":"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7","Type":"ContainerStarted","Data":"a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f"} Jul 27 14:33:31.807246 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.807087 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" event={"ID":"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7","Type":"ContainerStarted","Data":"d5b8fca462b40320a78488abc04e9972ab68813828cedf9bb027a25e02941efb"} Jul 27 14:33:31.822035 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:31.821984 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" podStartSLOduration=0.821968107 podStartE2EDuration="821.968107ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:33:31.820987217 +0000 UTC m=+995.188249264" watchObservedRunningTime="2026-07-27 14:33:31.821968107 +0000 UTC m=+995.189230154" Jul 27 14:33:36.625983 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.625940 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf"] Jul 27 14:33:36.632526 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.632486 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.636120 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.636085 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Jul 27 14:33:36.636389 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.636371 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-s8wmw\"" Jul 27 14:33:36.636642 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.636612 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Jul 27 14:33:36.636760 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.636622 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Jul 27 14:33:36.642837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.642810 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf"] Jul 27 14:33:36.742308 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.742274 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/30c3b990-ae2e-43a0-b788-e1629be6449f-feast-data\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.742490 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.742353 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-registry-tls\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.742490 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.742382 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-online-tls\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.742490 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.742416 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzpwf\" (UniqueName: \"kubernetes.io/projected/30c3b990-ae2e-43a0-b788-e1629be6449f-kube-api-access-tzpwf\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.843488 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.843456 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpwf\" (UniqueName: \"kubernetes.io/projected/30c3b990-ae2e-43a0-b788-e1629be6449f-kube-api-access-tzpwf\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.843640 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.843500 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/30c3b990-ae2e-43a0-b788-e1629be6449f-feast-data\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.843640 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.843571 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-registry-tls\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.843640 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.843608 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-online-tls\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.843962 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.843941 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/30c3b990-ae2e-43a0-b788-e1629be6449f-feast-data\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.846107 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.846084 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-online-tls\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.846107 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.846104 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-registry-tls\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.851346 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.851322 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzpwf\" (UniqueName: \"kubernetes.io/projected/30c3b990-ae2e-43a0-b788-e1629be6449f-kube-api-access-tzpwf\") pod \"feast-credit-scoring-567fdcf469-qzznf\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:36.944850 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:36.944796 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:37.087601 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:37.087567 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf"] Jul 27 14:33:37.090530 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:33:37.090502 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c3b990_ae2e_43a0_b788_e1629be6449f.slice/crio-4fc8d4a799df06f8844cea7baa6fe584d7a95ef2fd09975d27f32b83dfcf9092 WatchSource:0}: Error finding container 4fc8d4a799df06f8844cea7baa6fe584d7a95ef2fd09975d27f32b83dfcf9092: Status 404 returned error can't find the container with id 4fc8d4a799df06f8844cea7baa6fe584d7a95ef2fd09975d27f32b83dfcf9092 Jul 27 14:33:37.836127 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:37.836093 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerStarted","Data":"5a338b2798a4d0da81a3f60cd4de6b078de2f57b998e5f3e6efdaf12282d45c8"} Jul 27 14:33:37.836501 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:37.836142 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerStarted","Data":"4fc8d4a799df06f8844cea7baa6fe584d7a95ef2fd09975d27f32b83dfcf9092"} Jul 27 14:33:38.842826 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:38.842790 2555 generic.go:356] "Generic (PLEG): container finished" podID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerID="5a338b2798a4d0da81a3f60cd4de6b078de2f57b998e5f3e6efdaf12282d45c8" exitCode=0 Jul 27 14:33:38.843311 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:38.842857 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerDied","Data":"5a338b2798a4d0da81a3f60cd4de6b078de2f57b998e5f3e6efdaf12282d45c8"} Jul 27 14:33:39.852127 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:39.852095 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerStarted","Data":"17ec66697e05234cc4ce5ccf0884e5b24013c989af9af63c6045f6c0f1abab70"} Jul 27 14:33:45.883197 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:45.883164 2555 generic.go:356] "Generic (PLEG): container finished" podID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerID="17ec66697e05234cc4ce5ccf0884e5b24013c989af9af63c6045f6c0f1abab70" exitCode=0 Jul 27 14:33:45.883548 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:45.883238 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerDied","Data":"17ec66697e05234cc4ce5ccf0884e5b24013c989af9af63c6045f6c0f1abab70"} Jul 27 14:33:46.891918 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:46.891880 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerStarted","Data":"e7c3c8cba0d6d178a70c43b2959171f6447bc63ab4a7bc371551daa6e0419ea3"} Jul 27 14:33:46.892427 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:46.891966 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerStarted","Data":"2c363bd8a3a4e43064a5c14da05e03ac675736525989625a0317cde185c0ef54"} Jul 27 14:33:46.916035 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:46.915983 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podStartSLOduration=10.915966835 podStartE2EDuration="10.915966835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:33:46.913784725 +0000 UTC m=+1010.281046778" watchObservedRunningTime="2026-07-27 14:33:46.915966835 +0000 UTC m=+1010.283228881" Jul 27 14:33:46.945819 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:46.945784 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:46.945981 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:46.945846 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:48.945767 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:48.945728 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:48.946262 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:48.945801 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:48.947351 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:48.947323 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Jul 27 14:33:48.947446 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:48.947362 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 27 14:33:51.945886 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:51.945835 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Jul 27 14:33:51.946289 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:51.945851 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 27 14:33:54.945438 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:54.945397 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Jul 27 14:33:54.951212 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:54.951187 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:54.954471 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:54.954449 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:57.945922 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:57.945889 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:33:57.946858 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:33:57.946836 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:34:02.539206 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.539172 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27"] Jul 27 14:34:02.544038 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.544020 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.547343 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.547312 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jul 27 14:34:02.547343 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.547328 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Jul 27 14:34:02.547343 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.547328 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Jul 27 14:34:02.547553 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.547373 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-ms6gz\"" Jul 27 14:34:02.556921 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.556899 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27"] Jul 27 14:34:02.572343 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.572315 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-feast-data\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.572475 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.572430 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sj8d\" (UniqueName: \"kubernetes.io/projected/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-kube-api-access-5sj8d\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.572543 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.572520 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-registry-tls\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.572589 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.572542 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-online-tls\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.673920 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.673887 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-feast-data\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.674076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.673939 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sj8d\" (UniqueName: \"kubernetes.io/projected/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-kube-api-access-5sj8d\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.674076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.673999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-registry-tls\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.674076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.674017 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-online-tls\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.674342 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.674318 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-feast-data\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.676415 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.676396 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-online-tls\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.676480 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.676445 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-registry-tls\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.681965 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.681942 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sj8d\" (UniqueName: \"kubernetes.io/projected/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-kube-api-access-5sj8d\") pod \"feast-driver-ranking-65966b68cd-nxp27\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.854157 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.854077 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:02.991412 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:02.991384 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27"] Jul 27 14:34:02.993245 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:34:02.993216 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode833e0d7_a5cd_48bf_b763_62ec77b6b9b8.slice/crio-6392ad013e6d86096c3475312b086ffb498276dd13e3a7a82c5d9a8fd6959b8e WatchSource:0}: Error finding container 6392ad013e6d86096c3475312b086ffb498276dd13e3a7a82c5d9a8fd6959b8e: Status 404 returned error can't find the container with id 6392ad013e6d86096c3475312b086ffb498276dd13e3a7a82c5d9a8fd6959b8e Jul 27 14:34:03.979604 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:03.979570 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerStarted","Data":"e80637ebf474a71c3515e25e7114652fd03f3abebb23c4118bd68a7355223218"} Jul 27 14:34:03.980005 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:03.979620 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerStarted","Data":"6392ad013e6d86096c3475312b086ffb498276dd13e3a7a82c5d9a8fd6959b8e"} Jul 27 14:34:08.001692 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:08.001586 2555 generic.go:356] "Generic (PLEG): container finished" podID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerID="e80637ebf474a71c3515e25e7114652fd03f3abebb23c4118bd68a7355223218" exitCode=0 Jul 27 14:34:08.001692 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:08.001635 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerDied","Data":"e80637ebf474a71c3515e25e7114652fd03f3abebb23c4118bd68a7355223218"} Jul 27 14:34:09.008583 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:09.008550 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerStarted","Data":"ab9aeaf2abc90df04795fac7b8e99dadaa3303673dfbcefbbaebe32ff1a06a01"} Jul 27 14:34:15.038370 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:15.038335 2555 generic.go:356] "Generic (PLEG): container finished" podID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerID="ab9aeaf2abc90df04795fac7b8e99dadaa3303673dfbcefbbaebe32ff1a06a01" exitCode=0 Jul 27 14:34:15.038788 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:15.038409 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerDied","Data":"ab9aeaf2abc90df04795fac7b8e99dadaa3303673dfbcefbbaebe32ff1a06a01"} Jul 27 14:34:16.045985 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:16.045949 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerStarted","Data":"794eaced5e3ab073823ec8ee774036dcf103682ead9fafb50577786541d46fd6"} Jul 27 14:34:16.046372 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:16.046002 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerStarted","Data":"bc01f259c3d0458c35bbdc490bc83b6e2f2efb44d417bb0d18c6d74725287dab"} Jul 27 14:34:16.071872 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:16.071812 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podStartSLOduration=14.071790907 podStartE2EDuration="14.071790907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:34:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:34:16.068094779 +0000 UTC m=+1039.435356827" watchObservedRunningTime="2026-07-27 14:34:16.071790907 +0000 UTC m=+1039.439052959" Jul 27 14:34:17.855290 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:17.855249 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:17.855689 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:17.855330 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:17.856859 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:17.856826 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Jul 27 14:34:17.856994 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:17.856873 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Jul 27 14:34:20.854483 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:20.854433 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Jul 27 14:34:21.272857 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:21.272823 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:21.273316 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:21.273292 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:21.276384 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:21.276356 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:22.854215 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:22.854172 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:23.854851 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:23.854806 2555 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Jul 27 14:34:26.855263 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:26.855232 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:34:26.855944 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:34:26.855923 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:35:42.003843 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.003803 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw"] Jul 27 14:35:42.004348 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.004127 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" podUID="d54ce285-0716-426f-9c96-ce2ba6cab7be" containerName="manager" containerID="cri-o://29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f" gracePeriod=10 Jul 27 14:35:42.269996 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.269907 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf"] Jul 27 14:35:42.270718 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.270149 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="registry" containerID="cri-o://2c363bd8a3a4e43064a5c14da05e03ac675736525989625a0317cde185c0ef54" gracePeriod=30 Jul 27 14:35:42.270718 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.270277 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" containerID="cri-o://e7c3c8cba0d6d178a70c43b2959171f6447bc63ab4a7bc371551daa6e0419ea3" gracePeriod=30 Jul 27 14:35:42.277512 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.277483 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27"] Jul 27 14:35:42.277864 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.277812 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="registry" containerID="cri-o://bc01f259c3d0458c35bbdc490bc83b6e2f2efb44d417bb0d18c6d74725287dab" gracePeriod=30 Jul 27 14:35:42.277990 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.277829 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="online" containerID="cri-o://794eaced5e3ab073823ec8ee774036dcf103682ead9fafb50577786541d46fd6" gracePeriod=30 Jul 27 14:35:42.295325 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.295295 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:35:42.352626 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.352593 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg7nr\" (UniqueName: \"kubernetes.io/projected/d54ce285-0716-426f-9c96-ce2ba6cab7be-kube-api-access-zg7nr\") pod \"d54ce285-0716-426f-9c96-ce2ba6cab7be\" (UID: \"d54ce285-0716-426f-9c96-ce2ba6cab7be\") " Jul 27 14:35:42.354771 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.354746 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d54ce285-0716-426f-9c96-ce2ba6cab7be-kube-api-access-zg7nr" (OuterVolumeSpecName: "kube-api-access-zg7nr") pod "d54ce285-0716-426f-9c96-ce2ba6cab7be" (UID: "d54ce285-0716-426f-9c96-ce2ba6cab7be"). InnerVolumeSpecName "kube-api-access-zg7nr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:35:42.453559 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.453526 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg7nr\" (UniqueName: \"kubernetes.io/projected/d54ce285-0716-426f-9c96-ce2ba6cab7be-kube-api-access-zg7nr\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:35:42.474644 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.474609 2555 generic.go:356] "Generic (PLEG): container finished" podID="d54ce285-0716-426f-9c96-ce2ba6cab7be" containerID="29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f" exitCode=0 Jul 27 14:35:42.474841 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.474678 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" Jul 27 14:35:42.474841 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.474698 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" event={"ID":"d54ce285-0716-426f-9c96-ce2ba6cab7be","Type":"ContainerDied","Data":"29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f"} Jul 27 14:35:42.474841 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.474753 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw" event={"ID":"d54ce285-0716-426f-9c96-ce2ba6cab7be","Type":"ContainerDied","Data":"a3db96585fa4c57da4c205a1d1fc9e3e8205f12aa19bd6882c62955812b96df2"} Jul 27 14:35:42.474841 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.474777 2555 scope.go:117] "RemoveContainer" containerID="29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f" Jul 27 14:35:42.484616 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.484602 2555 scope.go:117] "RemoveContainer" containerID="29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f" Jul 27 14:35:42.484888 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:35:42.484871 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f\": container with ID starting with 29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f not found: ID does not exist" containerID="29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f" Jul 27 14:35:42.484969 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.484893 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f"} err="failed to get container status \"29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f\": rpc error: code = NotFound desc = could not find container \"29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f\": container with ID starting with 29ef7d33eaad084730f208321d05c71fde8942e48d42cd3ab130cef0a997848f not found: ID does not exist" Jul 27 14:35:42.497892 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.497869 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw"] Jul 27 14:35:42.501765 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:42.501743 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6598b8cfdf-8ngdw"] Jul 27 14:35:43.147751 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:43.147721 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d54ce285-0716-426f-9c96-ce2ba6cab7be" path="/var/lib/kubelet/pods/d54ce285-0716-426f-9c96-ce2ba6cab7be/volumes" Jul 27 14:35:43.484074 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:43.484042 2555 generic.go:356] "Generic (PLEG): container finished" podID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerID="794eaced5e3ab073823ec8ee774036dcf103682ead9fafb50577786541d46fd6" exitCode=0 Jul 27 14:35:43.484276 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:43.484124 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerDied","Data":"794eaced5e3ab073823ec8ee774036dcf103682ead9fafb50577786541d46fd6"} Jul 27 14:35:43.487929 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:43.487905 2555 generic.go:356] "Generic (PLEG): container finished" podID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerID="e7c3c8cba0d6d178a70c43b2959171f6447bc63ab4a7bc371551daa6e0419ea3" exitCode=0 Jul 27 14:35:43.488028 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:43.487939 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerDied","Data":"e7c3c8cba0d6d178a70c43b2959171f6447bc63ab4a7bc371551daa6e0419ea3"} Jul 27 14:35:44.742812 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:44.742776 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-85x5t"] Jul 27 14:35:44.743302 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:44.742939 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" podUID="bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7" containerName="postgres" containerID="cri-o://a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f" gracePeriod=30 Jul 27 14:35:44.747413 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:44.747389 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6tqvl"] Jul 27 14:35:44.747565 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:44.747546 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" podUID="21071ca9-242f-453d-9572-77f744c97d1f" containerName="redis" containerID="cri-o://bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2" gracePeriod=30 Jul 27 14:35:44.952319 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:44.952270 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 27 14:35:45.011782 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.011755 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:35:45.014471 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.014453 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" Jul 27 14:35:45.075262 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.075236 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlmzs\" (UniqueName: \"kubernetes.io/projected/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-kube-api-access-rlmzs\") pod \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\" (UID: \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\") " Jul 27 14:35:45.075419 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.075305 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgmpt\" (UniqueName: \"kubernetes.io/projected/21071ca9-242f-453d-9572-77f744c97d1f-kube-api-access-fgmpt\") pod \"21071ca9-242f-453d-9572-77f744c97d1f\" (UID: \"21071ca9-242f-453d-9572-77f744c97d1f\") " Jul 27 14:35:45.075419 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.075330 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-postgresdata\") pod \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\" (UID: \"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7\") " Jul 27 14:35:45.077363 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.077338 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21071ca9-242f-453d-9572-77f744c97d1f-kube-api-access-fgmpt" (OuterVolumeSpecName: "kube-api-access-fgmpt") pod "21071ca9-242f-453d-9572-77f744c97d1f" (UID: "21071ca9-242f-453d-9572-77f744c97d1f"). InnerVolumeSpecName "kube-api-access-fgmpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:35:45.077511 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.077424 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-kube-api-access-rlmzs" (OuterVolumeSpecName: "kube-api-access-rlmzs") pod "bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7" (UID: "bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7"). InnerVolumeSpecName "kube-api-access-rlmzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:35:45.106850 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.106825 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7" (UID: "bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:35:45.176078 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.176052 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgmpt\" (UniqueName: \"kubernetes.io/projected/21071ca9-242f-453d-9572-77f744c97d1f-kube-api-access-fgmpt\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:35:45.176078 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.176076 2555 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-postgresdata\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:35:45.176210 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.176085 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlmzs\" (UniqueName: \"kubernetes.io/projected/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7-kube-api-access-rlmzs\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:35:45.498640 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.498609 2555 generic.go:356] "Generic (PLEG): container finished" podID="bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7" containerID="a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f" exitCode=0 Jul 27 14:35:45.498847 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.498651 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" Jul 27 14:35:45.498847 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.498703 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" event={"ID":"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7","Type":"ContainerDied","Data":"a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f"} Jul 27 14:35:45.498847 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.498743 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-85x5t" event={"ID":"bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7","Type":"ContainerDied","Data":"d5b8fca462b40320a78488abc04e9972ab68813828cedf9bb027a25e02941efb"} Jul 27 14:35:45.498847 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.498762 2555 scope.go:117] "RemoveContainer" containerID="a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f" Jul 27 14:35:45.500126 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.500101 2555 generic.go:356] "Generic (PLEG): container finished" podID="21071ca9-242f-453d-9572-77f744c97d1f" containerID="bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2" exitCode=0 Jul 27 14:35:45.500221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.500143 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" Jul 27 14:35:45.500221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.500156 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" event={"ID":"21071ca9-242f-453d-9572-77f744c97d1f","Type":"ContainerDied","Data":"bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2"} Jul 27 14:35:45.500221 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.500188 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-6tqvl" event={"ID":"21071ca9-242f-453d-9572-77f744c97d1f","Type":"ContainerDied","Data":"c6eec998ad52c767d1bc03b89a9d443f68226f831882dde86f029471cc551f23"} Jul 27 14:35:45.509533 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.509515 2555 scope.go:117] "RemoveContainer" containerID="a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f" Jul 27 14:35:45.509823 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:35:45.509805 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f\": container with ID starting with a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f not found: ID does not exist" containerID="a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f" Jul 27 14:35:45.509877 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.509832 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f"} err="failed to get container status \"a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f\": rpc error: code = NotFound desc = could not find container \"a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f\": container with ID starting with a55655bf4b77b79f0257ef4038869da1ef66526d0f6e641f6057fec9683cfd6f not found: ID does not exist" Jul 27 14:35:45.509877 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.509849 2555 scope.go:117] "RemoveContainer" containerID="bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2" Jul 27 14:35:45.517688 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.517653 2555 scope.go:117] "RemoveContainer" containerID="bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2" Jul 27 14:35:45.518133 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:35:45.518070 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2\": container with ID starting with bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2 not found: ID does not exist" containerID="bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2" Jul 27 14:35:45.518133 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.518108 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2"} err="failed to get container status \"bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2\": rpc error: code = NotFound desc = could not find container \"bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2\": container with ID starting with bd7fc8a17fede58c252aadce19aa97965a9772110815c471e96c83501e1c65e2 not found: ID does not exist" Jul 27 14:35:45.518416 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.518397 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6tqvl"] Jul 27 14:35:45.519954 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.519936 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6tqvl"] Jul 27 14:35:45.530865 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.530846 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-85x5t"] Jul 27 14:35:45.536638 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:45.536618 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-85x5t"] Jul 27 14:35:47.147942 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:47.147897 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21071ca9-242f-453d-9572-77f744c97d1f" path="/var/lib/kubelet/pods/21071ca9-242f-453d-9572-77f744c97d1f/volumes" Jul 27 14:35:47.148445 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:47.148422 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7" path="/var/lib/kubelet/pods/bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7/volumes" Jul 27 14:35:51.274286 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:51.274246 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Jul 27 14:35:54.952543 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:54.952495 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 27 14:35:57.132823 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:57.132789 2555 scope.go:117] "RemoveContainer" containerID="939ae44881da8493d6e42bbc1f829b421241bdc2990f917fbabefde0015dd7d9" Jul 27 14:35:57.141382 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:57.141360 2555 scope.go:117] "RemoveContainer" containerID="660dbb7dde6c17ce390b43384f966e80eebc04b3c1c8412e2db5d55609ea4723" Jul 27 14:35:57.150307 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:57.150287 2555 scope.go:117] "RemoveContainer" containerID="9cdf4e830b336f534faf1250e87463a7a2d79b6fed4683307730bedcd31123f1" Jul 27 14:35:57.157836 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:57.157814 2555 scope.go:117] "RemoveContainer" containerID="0510744f0fa76693b060c457681d541849aba35d60f751db263aed686bc7a2b3" Jul 27 14:35:57.165595 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:35:57.165571 2555 scope.go:117] "RemoveContainer" containerID="4d7dba96d5ca602de4227ddbec9697735999cbc1ccb4970b7c1513971a49dfde" Jul 27 14:36:01.273734 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:01.273696 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Jul 27 14:36:04.952133 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:04.952092 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 27 14:36:04.952540 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:04.952209 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:36:11.273437 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:11.273397 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Jul 27 14:36:11.273988 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:11.273534 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:36:12.627688 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:12.627593 2555 generic.go:356] "Generic (PLEG): container finished" podID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerID="bc01f259c3d0458c35bbdc490bc83b6e2f2efb44d417bb0d18c6d74725287dab" exitCode=137 Jul 27 14:36:12.628088 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:12.627680 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerDied","Data":"bc01f259c3d0458c35bbdc490bc83b6e2f2efb44d417bb0d18c6d74725287dab"} Jul 27 14:36:12.629944 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:12.629924 2555 generic.go:356] "Generic (PLEG): container finished" podID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerID="2c363bd8a3a4e43064a5c14da05e03ac675736525989625a0317cde185c0ef54" exitCode=137 Jul 27 14:36:12.630033 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:12.629960 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerDied","Data":"2c363bd8a3a4e43064a5c14da05e03ac675736525989625a0317cde185c0ef54"} Jul 27 14:36:12.947379 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:12.947356 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:36:12.950070 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:12.950051 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:36:13.030822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.030786 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-feast-data\") pod \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " Jul 27 14:36:13.030822 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.030830 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-registry-tls\") pod \"30c3b990-ae2e-43a0-b788-e1629be6449f\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " Jul 27 14:36:13.031076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.030863 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sj8d\" (UniqueName: \"kubernetes.io/projected/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-kube-api-access-5sj8d\") pod \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " Jul 27 14:36:13.031076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.030909 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-registry-tls\") pod \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " Jul 27 14:36:13.031076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.030959 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-online-tls\") pod \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\" (UID: \"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8\") " Jul 27 14:36:13.031076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.030977 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/30c3b990-ae2e-43a0-b788-e1629be6449f-feast-data\") pod \"30c3b990-ae2e-43a0-b788-e1629be6449f\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " Jul 27 14:36:13.031076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.031005 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzpwf\" (UniqueName: \"kubernetes.io/projected/30c3b990-ae2e-43a0-b788-e1629be6449f-kube-api-access-tzpwf\") pod \"30c3b990-ae2e-43a0-b788-e1629be6449f\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " Jul 27 14:36:13.031076 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.031046 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-online-tls\") pod \"30c3b990-ae2e-43a0-b788-e1629be6449f\" (UID: \"30c3b990-ae2e-43a0-b788-e1629be6449f\") " Jul 27 14:36:13.031634 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.031453 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-feast-data" (OuterVolumeSpecName: "feast-data") pod "e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" (UID: "e833e0d7-a5cd-48bf-b763-62ec77b6b9b8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:36:13.033161 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.033132 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-kube-api-access-5sj8d" (OuterVolumeSpecName: "kube-api-access-5sj8d") pod "e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" (UID: "e833e0d7-a5cd-48bf-b763-62ec77b6b9b8"). InnerVolumeSpecName "kube-api-access-5sj8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:36:13.033511 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.033479 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-online-tls" (OuterVolumeSpecName: "online-tls") pod "e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" (UID: "e833e0d7-a5cd-48bf-b763-62ec77b6b9b8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:36:13.033623 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.033503 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" (UID: "e833e0d7-a5cd-48bf-b763-62ec77b6b9b8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:36:13.033623 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.033538 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-online-tls" (OuterVolumeSpecName: "online-tls") pod "30c3b990-ae2e-43a0-b788-e1629be6449f" (UID: "30c3b990-ae2e-43a0-b788-e1629be6449f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:36:13.033623 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.033555 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "30c3b990-ae2e-43a0-b788-e1629be6449f" (UID: "30c3b990-ae2e-43a0-b788-e1629be6449f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 14:36:13.033815 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.033653 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30c3b990-ae2e-43a0-b788-e1629be6449f-kube-api-access-tzpwf" (OuterVolumeSpecName: "kube-api-access-tzpwf") pod "30c3b990-ae2e-43a0-b788-e1629be6449f" (UID: "30c3b990-ae2e-43a0-b788-e1629be6449f"). InnerVolumeSpecName "kube-api-access-tzpwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:36:13.040908 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.040879 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30c3b990-ae2e-43a0-b788-e1629be6449f-feast-data" (OuterVolumeSpecName: "feast-data") pod "30c3b990-ae2e-43a0-b788-e1629be6449f" (UID: "30c3b990-ae2e-43a0-b788-e1629be6449f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:36:13.131946 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.131911 2555 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-online-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:36:13.131946 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.131942 2555 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/30c3b990-ae2e-43a0-b788-e1629be6449f-feast-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:36:13.131946 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.131951 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzpwf\" (UniqueName: \"kubernetes.io/projected/30c3b990-ae2e-43a0-b788-e1629be6449f-kube-api-access-tzpwf\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:36:13.132191 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.131961 2555 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-online-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:36:13.132191 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.131970 2555 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-feast-data\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:36:13.132191 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.131978 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/30c3b990-ae2e-43a0-b788-e1629be6449f-registry-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:36:13.132191 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.131985 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sj8d\" (UniqueName: \"kubernetes.io/projected/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-kube-api-access-5sj8d\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:36:13.132191 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.131993 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8-registry-tls\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:36:13.636781 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.636691 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" Jul 27 14:36:13.636781 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.636692 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf" event={"ID":"30c3b990-ae2e-43a0-b788-e1629be6449f","Type":"ContainerDied","Data":"4fc8d4a799df06f8844cea7baa6fe584d7a95ef2fd09975d27f32b83dfcf9092"} Jul 27 14:36:13.637364 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.636836 2555 scope.go:117] "RemoveContainer" containerID="e7c3c8cba0d6d178a70c43b2959171f6447bc63ab4a7bc371551daa6e0419ea3" Jul 27 14:36:13.639393 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.639368 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" event={"ID":"e833e0d7-a5cd-48bf-b763-62ec77b6b9b8","Type":"ContainerDied","Data":"6392ad013e6d86096c3475312b086ffb498276dd13e3a7a82c5d9a8fd6959b8e"} Jul 27 14:36:13.639509 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.639390 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27" Jul 27 14:36:13.646351 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.646335 2555 scope.go:117] "RemoveContainer" containerID="2c363bd8a3a4e43064a5c14da05e03ac675736525989625a0317cde185c0ef54" Jul 27 14:36:13.653997 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.653975 2555 scope.go:117] "RemoveContainer" containerID="17ec66697e05234cc4ce5ccf0884e5b24013c989af9af63c6045f6c0f1abab70" Jul 27 14:36:13.655882 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.655861 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27"] Jul 27 14:36:13.659097 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.659078 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65966b68cd-nxp27"] Jul 27 14:36:13.661759 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.661742 2555 scope.go:117] "RemoveContainer" containerID="5a338b2798a4d0da81a3f60cd4de6b078de2f57b998e5f3e6efdaf12282d45c8" Jul 27 14:36:13.669547 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.669529 2555 scope.go:117] "RemoveContainer" containerID="794eaced5e3ab073823ec8ee774036dcf103682ead9fafb50577786541d46fd6" Jul 27 14:36:13.669925 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.669903 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf"] Jul 27 14:36:13.676040 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.676018 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-567fdcf469-qzznf"] Jul 27 14:36:13.677913 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.677894 2555 scope.go:117] "RemoveContainer" containerID="bc01f259c3d0458c35bbdc490bc83b6e2f2efb44d417bb0d18c6d74725287dab" Jul 27 14:36:13.685629 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.685611 2555 scope.go:117] "RemoveContainer" containerID="ab9aeaf2abc90df04795fac7b8e99dadaa3303673dfbcefbbaebe32ff1a06a01" Jul 27 14:36:13.692827 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:13.692809 2555 scope.go:117] "RemoveContainer" containerID="e80637ebf474a71c3515e25e7114652fd03f3abebb23c4118bd68a7355223218" Jul 27 14:36:15.147463 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:15.147428 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" path="/var/lib/kubelet/pods/30c3b990-ae2e-43a0-b788-e1629be6449f/volumes" Jul 27 14:36:15.147973 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:15.147958 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" path="/var/lib/kubelet/pods/e833e0d7-a5cd-48bf-b763-62ec77b6b9b8/volumes" Jul 27 14:36:57.096736 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:57.096705 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:36:57.098680 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:36:57.098646 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:41:57.128809 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:41:57.128775 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:41:57.129376 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:41:57.129361 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:46:36.510643 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510562 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vxzr4/must-gather-5l24s"] Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510893 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="registry" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510905 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="registry" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510917 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="feast-init" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510922 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="feast-init" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510933 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="registry" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510938 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="registry" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510944 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="online" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510950 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="online" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510958 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="feast-init" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510964 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="feast-init" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510970 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510975 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510983 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="feast-apply" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510989 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="feast-apply" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.510996 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d54ce285-0716-426f-9c96-ce2ba6cab7be" containerName="manager" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511000 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d54ce285-0716-426f-9c96-ce2ba6cab7be" containerName="manager" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511007 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="feast-apply" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511012 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="feast-apply" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511020 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21071ca9-242f-453d-9572-77f744c97d1f" containerName="redis" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511025 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="21071ca9-242f-453d-9572-77f744c97d1f" containerName="redis" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511032 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7" containerName="postgres" Jul 27 14:46:36.511054 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511038 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7" containerName="postgres" Jul 27 14:46:36.511740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511084 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="21071ca9-242f-453d-9572-77f744c97d1f" containerName="redis" Jul 27 14:46:36.511740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511092 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="online" Jul 27 14:46:36.511740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511101 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="registry" Jul 27 14:46:36.511740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511107 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e833e0d7-a5cd-48bf-b763-62ec77b6b9b8" containerName="registry" Jul 27 14:46:36.511740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511113 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c3b990-ae2e-43a0-b788-e1629be6449f" containerName="online" Jul 27 14:46:36.511740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511118 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc65dd0d-87c8-4a37-bc34-b6b7dcaf98a7" containerName="postgres" Jul 27 14:46:36.511740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.511125 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d54ce285-0716-426f-9c96-ce2ba6cab7be" containerName="manager" Jul 27 14:46:36.514109 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.514092 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:36.516703 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.516679 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vxzr4\"/\"default-dockercfg-zrdsv\"" Jul 27 14:46:36.516820 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.516690 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vxzr4\"/\"kube-root-ca.crt\"" Jul 27 14:46:36.517830 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.517810 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vxzr4\"/\"openshift-service-ca.crt\"" Jul 27 14:46:36.521734 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.521712 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vxzr4/must-gather-5l24s"] Jul 27 14:46:36.677591 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.677545 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/faed0046-5856-4d04-98d8-d2666d09a166-must-gather-output\") pod \"must-gather-5l24s\" (UID: \"faed0046-5856-4d04-98d8-d2666d09a166\") " pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:36.677806 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.677615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6knp9\" (UniqueName: \"kubernetes.io/projected/faed0046-5856-4d04-98d8-d2666d09a166-kube-api-access-6knp9\") pod \"must-gather-5l24s\" (UID: \"faed0046-5856-4d04-98d8-d2666d09a166\") " pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:36.779163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.779053 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/faed0046-5856-4d04-98d8-d2666d09a166-must-gather-output\") pod \"must-gather-5l24s\" (UID: \"faed0046-5856-4d04-98d8-d2666d09a166\") " pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:36.779163 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.779138 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6knp9\" (UniqueName: \"kubernetes.io/projected/faed0046-5856-4d04-98d8-d2666d09a166-kube-api-access-6knp9\") pod \"must-gather-5l24s\" (UID: \"faed0046-5856-4d04-98d8-d2666d09a166\") " pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:36.779408 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.779387 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/faed0046-5856-4d04-98d8-d2666d09a166-must-gather-output\") pod \"must-gather-5l24s\" (UID: \"faed0046-5856-4d04-98d8-d2666d09a166\") " pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:36.788913 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.788889 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6knp9\" (UniqueName: \"kubernetes.io/projected/faed0046-5856-4d04-98d8-d2666d09a166-kube-api-access-6knp9\") pod \"must-gather-5l24s\" (UID: \"faed0046-5856-4d04-98d8-d2666d09a166\") " pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:36.837993 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.837960 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:36.972635 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.972611 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vxzr4/must-gather-5l24s"] Jul 27 14:46:36.975121 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:46:36.975096 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfaed0046_5856_4d04_98d8_d2666d09a166.slice/crio-8cdb065cadf3e87fa99c5d45b1d0d3a6938c4de8123becf4fd5b4590781c0d74 WatchSource:0}: Error finding container 8cdb065cadf3e87fa99c5d45b1d0d3a6938c4de8123becf4fd5b4590781c0d74: Status 404 returned error can't find the container with id 8cdb065cadf3e87fa99c5d45b1d0d3a6938c4de8123becf4fd5b4590781c0d74 Jul 27 14:46:36.976828 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:36.976812 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 14:46:37.124186 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:37.124093 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vxzr4/must-gather-5l24s" event={"ID":"faed0046-5856-4d04-98d8-d2666d09a166","Type":"ContainerStarted","Data":"8cdb065cadf3e87fa99c5d45b1d0d3a6938c4de8123becf4fd5b4590781c0d74"} Jul 27 14:46:44.162362 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:44.162322 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vxzr4/must-gather-5l24s" event={"ID":"faed0046-5856-4d04-98d8-d2666d09a166","Type":"ContainerStarted","Data":"711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7"} Jul 27 14:46:44.162837 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:44.162373 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vxzr4/must-gather-5l24s" event={"ID":"faed0046-5856-4d04-98d8-d2666d09a166","Type":"ContainerStarted","Data":"bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e"} Jul 27 14:46:44.181344 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:44.181292 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vxzr4/must-gather-5l24s" podStartSLOduration=1.932302551 podStartE2EDuration="8.181273875s" totalImagesPullingTime="6.248971324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 14:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:46:44.179804475 +0000 UTC m=+1787.547066525" watchObservedRunningTime="2026-07-27 14:46:44.181273875 +0000 UTC m=+1787.548535923" Jul 27 14:46:53.203087 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:53.203051 2555 generic.go:356] "Generic (PLEG): container finished" podID="faed0046-5856-4d04-98d8-d2666d09a166" containerID="bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e" exitCode=0 Jul 27 14:46:53.203635 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:53.203109 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vxzr4/must-gather-5l24s" event={"ID":"faed0046-5856-4d04-98d8-d2666d09a166","Type":"ContainerDied","Data":"bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e"} Jul 27 14:46:53.203635 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:53.203505 2555 scope.go:117] "RemoveContainer" containerID="bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e" Jul 27 14:46:54.062962 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:54.062925 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vxzr4_must-gather-5l24s_faed0046-5856-4d04-98d8-d2666d09a166/gather/0.log" Jul 27 14:46:57.167035 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:57.167003 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:46:57.168393 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:57.168372 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:46:57.909879 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:57.909843 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-np467_9b27844f-7fba-4449-b514-1f2d9424f97e/global-pull-secret-syncer/0.log" Jul 27 14:46:58.054141 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:58.054103 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k696n_4b2d36d1-a3d2-477c-b22a-d7963daf4fb1/konnectivity-agent/0.log" Jul 27 14:46:58.107093 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:58.107063 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-223.ec2.internal_91823a60abd6f5026e7827f004ecd18b/haproxy/0.log" Jul 27 14:46:59.544158 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.544122 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-vxzr4/must-gather-5l24s"] Jul 27 14:46:59.544742 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.544378 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-vxzr4/must-gather-5l24s" podUID="faed0046-5856-4d04-98d8-d2666d09a166" containerName="copy" containerID="cri-o://711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7" gracePeriod=2 Jul 27 14:46:59.549713 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.549689 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-vxzr4/must-gather-5l24s"] Jul 27 14:46:59.776740 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.776716 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vxzr4_must-gather-5l24s_faed0046-5856-4d04-98d8-d2666d09a166/copy/0.log" Jul 27 14:46:59.779291 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.779270 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:59.781760 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:46:59.781733 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-5l24s\" is forbidden: User \"system:node:ip-10-0-133-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vxzr4\": no relationship found between node 'ip-10-0-133-223.ec2.internal' and this object" podUID="faed0046-5856-4d04-98d8-d2666d09a166" pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:46:59.882277 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.882186 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/faed0046-5856-4d04-98d8-d2666d09a166-must-gather-output\") pod \"faed0046-5856-4d04-98d8-d2666d09a166\" (UID: \"faed0046-5856-4d04-98d8-d2666d09a166\") " Jul 27 14:46:59.882277 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.882240 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6knp9\" (UniqueName: \"kubernetes.io/projected/faed0046-5856-4d04-98d8-d2666d09a166-kube-api-access-6knp9\") pod \"faed0046-5856-4d04-98d8-d2666d09a166\" (UID: \"faed0046-5856-4d04-98d8-d2666d09a166\") " Jul 27 14:46:59.882639 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.882613 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faed0046-5856-4d04-98d8-d2666d09a166-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "faed0046-5856-4d04-98d8-d2666d09a166" (UID: "faed0046-5856-4d04-98d8-d2666d09a166"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 14:46:59.884467 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.884434 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faed0046-5856-4d04-98d8-d2666d09a166-kube-api-access-6knp9" (OuterVolumeSpecName: "kube-api-access-6knp9") pod "faed0046-5856-4d04-98d8-d2666d09a166" (UID: "faed0046-5856-4d04-98d8-d2666d09a166"). InnerVolumeSpecName "kube-api-access-6knp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 14:46:59.983342 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.983301 2555 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/faed0046-5856-4d04-98d8-d2666d09a166-must-gather-output\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:46:59.983342 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:46:59.983336 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6knp9\" (UniqueName: \"kubernetes.io/projected/faed0046-5856-4d04-98d8-d2666d09a166-kube-api-access-6knp9\") on node \"ip-10-0-133-223.ec2.internal\" DevicePath \"\"" Jul 27 14:47:00.232589 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.232561 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vxzr4_must-gather-5l24s_faed0046-5856-4d04-98d8-d2666d09a166/copy/0.log" Jul 27 14:47:00.232927 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.232904 2555 generic.go:356] "Generic (PLEG): container finished" podID="faed0046-5856-4d04-98d8-d2666d09a166" containerID="711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7" exitCode=143 Jul 27 14:47:00.233035 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.232953 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:47:00.233035 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.232990 2555 scope.go:117] "RemoveContainer" containerID="711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7" Jul 27 14:47:00.235546 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:47:00.235524 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-5l24s\" is forbidden: User \"system:node:ip-10-0-133-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vxzr4\": no relationship found between node 'ip-10-0-133-223.ec2.internal' and this object" podUID="faed0046-5856-4d04-98d8-d2666d09a166" pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:47:00.241364 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.241346 2555 scope.go:117] "RemoveContainer" containerID="bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e" Jul 27 14:47:00.243414 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:47:00.243391 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-5l24s\" is forbidden: User \"system:node:ip-10-0-133-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vxzr4\": no relationship found between node 'ip-10-0-133-223.ec2.internal' and this object" podUID="faed0046-5856-4d04-98d8-d2666d09a166" pod="openshift-must-gather-vxzr4/must-gather-5l24s" Jul 27 14:47:00.252374 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.252357 2555 scope.go:117] "RemoveContainer" containerID="711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7" Jul 27 14:47:00.252671 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:47:00.252638 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7\": container with ID starting with 711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7 not found: ID does not exist" containerID="711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7" Jul 27 14:47:00.252725 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.252688 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7"} err="failed to get container status \"711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7\": rpc error: code = NotFound desc = could not find container \"711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7\": container with ID starting with 711f1a059745895d774f955d75896d6f2e029c9e11fb2c8b1b3fbb0b247479b7 not found: ID does not exist" Jul 27 14:47:00.252725 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.252706 2555 scope.go:117] "RemoveContainer" containerID="bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e" Jul 27 14:47:00.252932 ip-10-0-133-223 kubenswrapper[2555]: E0727 14:47:00.252913 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e\": container with ID starting with bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e not found: ID does not exist" containerID="bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e" Jul 27 14:47:00.252986 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:00.252942 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e"} err="failed to get container status \"bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e\": rpc error: code = NotFound desc = could not find container \"bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e\": container with ID starting with bde40fa568bb7a94819b671d64071a611a7f406179cc3a412431538660ea3e2e not found: ID does not exist" Jul 27 14:47:01.147679 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.147630 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faed0046-5856-4d04-98d8-d2666d09a166" path="/var/lib/kubelet/pods/faed0046-5856-4d04-98d8-d2666d09a166/volumes" Jul 27 14:47:01.494300 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.494262 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c6e8899c-f0ac-49ce-be97-ff98b9e55393/alertmanager/0.log" Jul 27 14:47:01.531565 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.531534 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c6e8899c-f0ac-49ce-be97-ff98b9e55393/config-reloader/0.log" Jul 27 14:47:01.563362 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.563328 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c6e8899c-f0ac-49ce-be97-ff98b9e55393/kube-rbac-proxy-web/0.log" Jul 27 14:47:01.596138 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.596112 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c6e8899c-f0ac-49ce-be97-ff98b9e55393/kube-rbac-proxy/0.log" Jul 27 14:47:01.623596 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.623567 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c6e8899c-f0ac-49ce-be97-ff98b9e55393/kube-rbac-proxy-metric/0.log" Jul 27 14:47:01.658129 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.658103 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c6e8899c-f0ac-49ce-be97-ff98b9e55393/prom-label-proxy/0.log" Jul 27 14:47:01.701078 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.701051 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c6e8899c-f0ac-49ce-be97-ff98b9e55393/init-config-reloader/0.log" Jul 27 14:47:01.747736 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.747648 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-fwmw7_028ffee0-b008-47b4-93b0-996dfbd4f7b1/cluster-monitoring-operator/0.log" Jul 27 14:47:01.781454 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.781425 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7ppt5_793e6f62-0fe6-46ca-aae0-2a68e2000fd9/kube-state-metrics/0.log" Jul 27 14:47:01.809470 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.809439 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7ppt5_793e6f62-0fe6-46ca-aae0-2a68e2000fd9/kube-rbac-proxy-main/0.log" Jul 27 14:47:01.848896 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.848863 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7ppt5_793e6f62-0fe6-46ca-aae0-2a68e2000fd9/kube-rbac-proxy-self/0.log" Jul 27 14:47:01.896835 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:01.896789 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b5f6b554c-46flw_268d2adf-43d2-4ab0-ae70-f86855843ab9/metrics-server/0.log" Jul 27 14:47:02.335798 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:02.335770 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-txldj_0f366d0f-4f55-4753-be5e-9e781463cc22/node-exporter/0.log" Jul 27 14:47:02.381325 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:02.381296 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-txldj_0f366d0f-4f55-4753-be5e-9e781463cc22/kube-rbac-proxy/0.log" Jul 27 14:47:02.423130 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:02.423106 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-txldj_0f366d0f-4f55-4753-be5e-9e781463cc22/init-textfile/0.log" Jul 27 14:47:03.109192 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.109165 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7576cf7d6-rrbvb_ffe40e22-cb73-4899-a1e1-190f1ad1a104/telemeter-client/0.log" Jul 27 14:47:03.175725 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.175703 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7576cf7d6-rrbvb_ffe40e22-cb73-4899-a1e1-190f1ad1a104/reload/0.log" Jul 27 14:47:03.274363 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.274334 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7576cf7d6-rrbvb_ffe40e22-cb73-4899-a1e1-190f1ad1a104/kube-rbac-proxy/0.log" Jul 27 14:47:03.370424 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.370335 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dfbfbd66c-pfzwf_69f241b0-2404-441a-93f7-97857c9b2da8/thanos-query/0.log" Jul 27 14:47:03.421706 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.421677 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dfbfbd66c-pfzwf_69f241b0-2404-441a-93f7-97857c9b2da8/kube-rbac-proxy-web/0.log" Jul 27 14:47:03.496890 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.496856 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dfbfbd66c-pfzwf_69f241b0-2404-441a-93f7-97857c9b2da8/kube-rbac-proxy/0.log" Jul 27 14:47:03.546511 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.546482 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dfbfbd66c-pfzwf_69f241b0-2404-441a-93f7-97857c9b2da8/prom-label-proxy/0.log" Jul 27 14:47:03.584681 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.584635 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dfbfbd66c-pfzwf_69f241b0-2404-441a-93f7-97857c9b2da8/kube-rbac-proxy-rules/0.log" Jul 27 14:47:03.626279 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:03.626191 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dfbfbd66c-pfzwf_69f241b0-2404-441a-93f7-97857c9b2da8/kube-rbac-proxy-metrics/0.log" Jul 27 14:47:05.239451 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.239419 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/1.log" Jul 27 14:47:05.243293 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.243260 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhx74_db5170c1-44f1-4ee2-a373-78788f49e622/console-operator/2.log" Jul 27 14:47:05.549339 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.549252 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd"] Jul 27 14:47:05.549696 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.549679 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faed0046-5856-4d04-98d8-d2666d09a166" containerName="copy" Jul 27 14:47:05.549796 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.549699 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="faed0046-5856-4d04-98d8-d2666d09a166" containerName="copy" Jul 27 14:47:05.549796 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.549739 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faed0046-5856-4d04-98d8-d2666d09a166" containerName="gather" Jul 27 14:47:05.549796 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.549749 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="faed0046-5856-4d04-98d8-d2666d09a166" containerName="gather" Jul 27 14:47:05.549952 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.549839 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="faed0046-5856-4d04-98d8-d2666d09a166" containerName="gather" Jul 27 14:47:05.549952 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.549854 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="faed0046-5856-4d04-98d8-d2666d09a166" containerName="copy" Jul 27 14:47:05.554642 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.554622 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.557803 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.557781 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbxjx\"/\"openshift-service-ca.crt\"" Jul 27 14:47:05.558039 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.558024 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbxjx\"/\"kube-root-ca.crt\"" Jul 27 14:47:05.558965 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.558948 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rbxjx\"/\"default-dockercfg-p8wn4\"" Jul 27 14:47:05.564933 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.564914 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd"] Jul 27 14:47:05.629467 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.629436 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-podres\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.629467 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.629471 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw5cn\" (UniqueName: \"kubernetes.io/projected/dfe71315-8d54-4287-a67f-3384896edad6-kube-api-access-rw5cn\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.629706 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.629507 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-sys\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.629706 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.629610 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-lib-modules\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.629706 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.629674 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-proc\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.730801 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.730767 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-podres\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.730801 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.730803 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw5cn\" (UniqueName: \"kubernetes.io/projected/dfe71315-8d54-4287-a67f-3384896edad6-kube-api-access-rw5cn\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.731065 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.730844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-sys\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.731065 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.730891 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-lib-modules\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.731065 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.730921 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-proc\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.731065 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.730931 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-podres\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.731065 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.730974 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-sys\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.731065 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.730997 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-proc\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.731065 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.731027 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfe71315-8d54-4287-a67f-3384896edad6-lib-modules\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.742185 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.742161 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw5cn\" (UniqueName: \"kubernetes.io/projected/dfe71315-8d54-4287-a67f-3384896edad6-kube-api-access-rw5cn\") pod \"perf-node-gather-daemonset-phxwd\" (UID: \"dfe71315-8d54-4287-a67f-3384896edad6\") " pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.865238 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.865123 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:05.996554 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:05.996521 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd"] Jul 27 14:47:05.999854 ip-10-0-133-223 kubenswrapper[2555]: W0727 14:47:05.999824 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddfe71315_8d54_4287_a67f_3384896edad6.slice/crio-7bbd38acd51f814e4d221804de9fa6b32d253c25fa323669084e2b78d5e95804 WatchSource:0}: Error finding container 7bbd38acd51f814e4d221804de9fa6b32d253c25fa323669084e2b78d5e95804: Status 404 returned error can't find the container with id 7bbd38acd51f814e4d221804de9fa6b32d253c25fa323669084e2b78d5e95804 Jul 27 14:47:06.202948 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:06.202917 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-z4nfd_7678c3a4-0414-4678-ade4-767be3864a91/volume-data-source-validator/0.log" Jul 27 14:47:06.259922 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:06.259875 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" event={"ID":"dfe71315-8d54-4287-a67f-3384896edad6","Type":"ContainerStarted","Data":"563d2fcf21cbed9ea88f5d82ad06cfdc213977b6f718c393028f9a695163003a"} Jul 27 14:47:06.260316 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:06.259968 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" event={"ID":"dfe71315-8d54-4287-a67f-3384896edad6","Type":"ContainerStarted","Data":"7bbd38acd51f814e4d221804de9fa6b32d253c25fa323669084e2b78d5e95804"} Jul 27 14:47:06.260316 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:06.260026 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:06.277540 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:06.277492 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" podStartSLOduration=1.277476538 podStartE2EDuration="1.277476538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 14:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 14:47:06.276577136 +0000 UTC m=+1809.643839183" watchObservedRunningTime="2026-07-27 14:47:06.277476538 +0000 UTC m=+1809.644738611" Jul 27 14:47:06.991692 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:06.991651 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8cbvq_00a47fc4-008e-4b66-8ae2-c5fed963599a/dns/0.log" Jul 27 14:47:07.019771 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:07.019744 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8cbvq_00a47fc4-008e-4b66-8ae2-c5fed963599a/kube-rbac-proxy/0.log" Jul 27 14:47:07.234503 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:07.234469 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jbjwv_4bdecfbd-3e2f-47f0-a85b-2efffb7aef68/dns-node-resolver/0.log" Jul 27 14:47:07.738528 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:07.738498 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gpfpw_def06289-55b1-475f-a338-42014932d174/node-ca/0.log" Jul 27 14:47:08.575610 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:08.575582 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7745b5dc69-b8hmd_c5d14ecc-98e1-45cf-9071-0106bec58113/router/0.log" Jul 27 14:47:08.965258 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:08.965227 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bwz8r_f0dfb067-bcb2-4d4c-88be-f93bec24022f/serve-healthcheck-canary/0.log" Jul 27 14:47:09.359233 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:09.359148 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-wdpj7_76fb3e7b-bef0-473d-a98e-0cbac98c891c/insights-operator/0.log" Jul 27 14:47:09.359406 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:09.359386 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-wdpj7_76fb3e7b-bef0-473d-a98e-0cbac98c891c/insights-operator/1.log" Jul 27 14:47:09.495998 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:09.495969 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssv66_7a7e56f1-e4f8-41da-afff-5b04280cb26e/kube-rbac-proxy/0.log" Jul 27 14:47:09.539984 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:09.539957 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssv66_7a7e56f1-e4f8-41da-afff-5b04280cb26e/exporter/0.log" Jul 27 14:47:09.593584 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:09.593558 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssv66_7a7e56f1-e4f8-41da-afff-5b04280cb26e/extractor/0.log" Jul 27 14:47:12.273783 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:12.273756 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rbxjx/perf-node-gather-daemonset-phxwd" Jul 27 14:47:14.590626 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:14.590589 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s_f86b19c8-ef8a-4e72-8048-c9fe8112d3ec/kube-storage-version-migrator-operator/1.log" Jul 27 14:47:14.591470 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:14.591451 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-pvn9s_f86b19c8-ef8a-4e72-8048-c9fe8112d3ec/kube-storage-version-migrator-operator/0.log" Jul 27 14:47:15.531846 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:15.531770 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-87smn_acf16aa1-2fc6-4275-9845-a26701408128/kube-multus-additional-cni-plugins/0.log" Jul 27 14:47:15.562111 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:15.562083 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-87smn_acf16aa1-2fc6-4275-9845-a26701408128/egress-router-binary-copy/0.log" Jul 27 14:47:15.590290 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:15.590263 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-87smn_acf16aa1-2fc6-4275-9845-a26701408128/cni-plugins/0.log" Jul 27 14:47:15.618027 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:15.618006 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-87smn_acf16aa1-2fc6-4275-9845-a26701408128/bond-cni-plugin/0.log" Jul 27 14:47:15.652150 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:15.652125 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-87smn_acf16aa1-2fc6-4275-9845-a26701408128/routeoverride-cni/0.log" Jul 27 14:47:15.678712 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:15.678680 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-87smn_acf16aa1-2fc6-4275-9845-a26701408128/whereabouts-cni-bincopy/0.log" Jul 27 14:47:15.703176 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:15.703143 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-87smn_acf16aa1-2fc6-4275-9845-a26701408128/whereabouts-cni/0.log" Jul 27 14:47:16.096809 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:16.096781 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-crs6s_7272bcef-4347-40ac-9ce4-9de68958d689/kube-multus/0.log" Jul 27 14:47:16.261126 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:16.261094 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lxpdl_a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de/network-metrics-daemon/0.log" Jul 27 14:47:16.286782 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:16.286755 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lxpdl_a5f4ef8c-fc0a-4e3a-8e64-7c54a3dd19de/kube-rbac-proxy/0.log" Jul 27 14:47:17.597050 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:17.597020 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xzr9g_38423966-1ba9-464f-8227-256adcb6b870/ovn-controller/0.log" Jul 27 14:47:17.616865 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:17.616825 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xzr9g_38423966-1ba9-464f-8227-256adcb6b870/ovn-acl-logging/0.log" Jul 27 14:47:17.644460 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:17.644436 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xzr9g_38423966-1ba9-464f-8227-256adcb6b870/kube-rbac-proxy-node/0.log" Jul 27 14:47:17.669462 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:17.669430 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xzr9g_38423966-1ba9-464f-8227-256adcb6b870/kube-rbac-proxy-ovn-metrics/0.log" Jul 27 14:47:17.691192 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:17.691139 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xzr9g_38423966-1ba9-464f-8227-256adcb6b870/northd/0.log" Jul 27 14:47:17.716881 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:17.716852 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xzr9g_38423966-1ba9-464f-8227-256adcb6b870/nbdb/0.log" Jul 27 14:47:17.740322 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:17.740300 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xzr9g_38423966-1ba9-464f-8227-256adcb6b870/sbdb/0.log" Jul 27 14:47:17.791237 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:17.791196 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xzr9g_38423966-1ba9-464f-8227-256adcb6b870/ovnkube-controller/0.log" Jul 27 14:47:18.771624 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:18.771596 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-4fqqw_723b1610-1571-4007-8a24-6451d3bf2153/check-endpoints/0.log" Jul 27 14:47:18.859336 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:18.859309 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-tl44d_0b1c2c00-b265-42b8-b25c-28a4acd4f3c7/network-check-target-container/0.log" Jul 27 14:47:19.794801 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:19.794773 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-nqrnp_d78abc26-4e7e-4509-befb-3e19f7dc9b3e/iptables-alerter/0.log" Jul 27 14:47:20.419760 ip-10-0-133-223 kubenswrapper[2555]: I0727 14:47:20.419733 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-fvnrd_eb8cb916-12e7-40eb-a9ce-2adf3ff30ca3/tuned/0.log"