Jul 14 19:37:56.387689 ip-10-0-128-126 systemd[1]: Starting Kubernetes Kubelet... Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.848163 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856282 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856297 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856301 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856305 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856308 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856312 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856317 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 19:37:56.935166 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856320 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856323 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856325 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856328 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856331 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856333 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856337 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856340 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856344 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856347 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856350 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856353 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856356 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856359 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856372 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856375 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856387 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856390 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856393 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 19:37:56.938080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856395 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 19:37:56.935205 ip-10-0-128-126 systemd[1]: Started Kubernetes Kubelet. Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856398 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856401 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856405 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856407 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856410 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856413 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856415 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856418 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856420 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856423 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856425 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856428 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856430 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856433 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856436 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856438 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856441 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856443 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856446 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 19:37:56.952313 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856448 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856451 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856453 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856456 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856459 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856462 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856464 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856467 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856470 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856473 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856475 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856478 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856480 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856483 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856486 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856489 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856492 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856494 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856497 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 19:37:56.968009 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856500 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856502 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856505 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856507 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856510 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856512 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856515 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856517 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856520 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856523 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856525 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856529 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856531 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856534 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856536 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856539 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856543 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856546 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856550 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856553 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 19:37:56.970655 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856556 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856558 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856564 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856567 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856570 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856572 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856575 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856578 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856581 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856584 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856586 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856589 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856592 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.856594 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858279 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858288 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858291 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858295 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858298 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 19:37:56.972789 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858313 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858317 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858321 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858323 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858326 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858329 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858332 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858334 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858337 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858340 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858342 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858345 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858347 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858350 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858353 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858355 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858364 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858366 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858369 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 19:37:56.975299 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858371 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858374 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858394 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858397 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858399 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858402 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858405 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858408 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858411 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858414 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858416 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858421 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858424 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858427 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858430 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858435 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858437 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858440 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858443 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858446 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 19:37:56.991979 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858448 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858451 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858455 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858459 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858462 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858465 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858468 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858471 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858473 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858476 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858527 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858743 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858751 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858756 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858761 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858766 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858777 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858781 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858785 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858790 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 19:37:57.147418 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858794 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858798 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858803 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858812 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858817 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858822 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858826 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858830 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858834 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858838 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858842 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858846 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858851 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858855 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858865 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858869 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858879 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858883 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858887 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858891 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 19:37:57.164016 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858895 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858899 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858905 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858941 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858947 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858951 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858956 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858960 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858969 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858973 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858978 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858982 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858986 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858990 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.858994 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860075 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860085 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860089 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860092 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860095 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 19:37:57.166892 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860098 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860101 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860104 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860107 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860110 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860113 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860115 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860118 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860122 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860125 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860127 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860130 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860132 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860135 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860138 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860141 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860143 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860146 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860149 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860151 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 19:37:57.169494 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860155 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860157 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860160 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860162 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860165 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860168 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860170 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860173 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860175 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860178 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860181 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860184 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860186 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860189 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860192 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860195 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860197 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860200 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860203 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860205 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 19:37:57.174361 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860208 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860211 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860213 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860216 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860218 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860221 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860223 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860226 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860228 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860231 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860234 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860237 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860239 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860242 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860245 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860248 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860251 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860253 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860256 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 19:37:57.176542 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860258 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860260 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860263 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860266 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860268 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860271 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860274 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860276 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860279 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860284 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860288 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860293 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860296 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860298 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860302 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860304 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860307 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860311 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860313 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 19:37:57.179196 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860318 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860323 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860327 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860331 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860334 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860338 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860340 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860343 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860346 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860349 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860351 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860355 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860357 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860360 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860363 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.860366 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860478 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860497 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860505 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860509 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860515 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 19:37:57.180430 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860518 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860524 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860528 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860531 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860535 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860539 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860542 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860545 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860548 2568 flags.go:64] FLAG: --cgroup-root="" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860551 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860554 2568 flags.go:64] FLAG: --client-ca-file="" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860557 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860561 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860565 2568 flags.go:64] FLAG: --cluster-domain="" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860568 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860571 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860574 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860578 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860582 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860585 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860588 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860591 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860594 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860597 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 19:37:57.181814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860600 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860603 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860606 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860610 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860613 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860616 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860620 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860623 2568 flags.go:64] FLAG: --enable-server="true" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860627 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860633 2568 flags.go:64] FLAG: --event-burst="100" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860636 2568 flags.go:64] FLAG: --event-qps="50" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860639 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860642 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860645 2568 flags.go:64] FLAG: --eviction-hard="" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860649 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860652 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860655 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860658 2568 flags.go:64] FLAG: --eviction-soft="" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860661 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860663 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860668 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860671 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860674 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860677 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860680 2568 flags.go:64] FLAG: --feature-gates="" Jul 14 19:37:57.182825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860684 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860687 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860690 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860693 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860696 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860699 2568 flags.go:64] FLAG: --help="false" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860702 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860706 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860708 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860711 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860715 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860718 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860721 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860724 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860726 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860729 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860734 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860737 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860740 2568 flags.go:64] FLAG: --kube-reserved="" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860743 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860746 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860749 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860751 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860754 2568 flags.go:64] FLAG: --lock-file="" Jul 14 19:37:57.184140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860757 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860760 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860763 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860770 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860773 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860776 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860779 2568 flags.go:64] FLAG: --logging-format="text" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860782 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860785 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860788 2568 flags.go:64] FLAG: --manifest-url="" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860791 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860795 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860798 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860803 2568 flags.go:64] FLAG: --max-pods="110" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860806 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860809 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860812 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860815 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860818 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860821 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860824 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860830 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860834 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860837 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 19:37:57.187712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860842 2568 flags.go:64] FLAG: --pod-cidr="" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860846 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860852 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860855 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860858 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860861 2568 flags.go:64] FLAG: --port="10250" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860864 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860867 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0168ae901de455fc2" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860870 2568 flags.go:64] FLAG: --qos-reserved="" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860873 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860876 2568 flags.go:64] FLAG: --register-node="true" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860879 2568 flags.go:64] FLAG: --register-with-taints="" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860883 2568 flags.go:64] FLAG: --registry-burst="10" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860887 2568 flags.go:64] FLAG: --registry-qps="5" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860889 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860892 2568 flags.go:64] FLAG: --reserved-memory="" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860896 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860899 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860902 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860905 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860907 2568 flags.go:64] FLAG: --runonce="false" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860910 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860913 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860916 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860919 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 19:37:57.188808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860922 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860925 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860928 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860931 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860934 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860937 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860939 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860945 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860948 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860952 2568 flags.go:64] FLAG: --system-cgroups="" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860954 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860960 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860963 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860966 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860970 2568 flags.go:64] FLAG: --tls-min-version="" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860972 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860975 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860978 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860981 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860985 2568 flags.go:64] FLAG: --v="2" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860989 2568 flags.go:64] FLAG: --version="false" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860993 2568 flags.go:64] FLAG: --vmodule="" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.860997 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.861000 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861120 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 19:37:57.194641 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861124 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861127 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861130 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861133 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861136 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861140 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861143 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861146 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861149 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861152 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861155 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861158 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861161 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861164 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861168 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861171 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861174 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861177 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861180 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861183 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 19:37:57.195365 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861186 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861189 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861191 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861194 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861196 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861199 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861203 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861205 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861208 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861211 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861213 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861216 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861218 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861221 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861224 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861226 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861229 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861231 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861234 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 19:37:57.195936 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861237 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861239 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861242 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861244 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861248 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861251 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861255 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861260 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861263 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861266 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861268 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861274 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861282 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861285 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861287 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861290 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861293 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861296 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861298 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 19:37:57.196565 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861304 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861306 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861309 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861312 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861314 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861317 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861319 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861322 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861324 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861327 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861329 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861332 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861334 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861337 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861340 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861342 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861345 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861348 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861350 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861353 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 19:37:57.197069 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861357 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861359 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861362 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861365 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861367 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861371 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861374 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861393 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861397 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861399 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861402 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861404 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861409 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861412 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861415 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861417 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861420 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861423 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861425 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 19:37:57.197786 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.861428 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.862124 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.869740 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.869763 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.869792 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.869804 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869888 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869894 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869899 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869904 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869909 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869913 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869917 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869922 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 19:37:57.198648 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869926 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869931 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869934 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869938 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869942 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869945 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869950 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869954 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869958 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869962 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869966 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869970 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869974 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869978 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869982 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869986 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869988 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869994 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.869998 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 19:37:57.199757 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870004 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870010 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870015 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870020 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870024 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870028 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870032 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870036 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870041 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870046 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870050 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870054 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870058 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870061 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870065 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870069 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870072 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870079 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870082 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870086 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 19:37:57.200622 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870091 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870095 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870099 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870103 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870106 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870110 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870114 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870118 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870121 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870125 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870132 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870137 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870141 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870144 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870148 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870152 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870156 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870160 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870164 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870167 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 19:37:57.201513 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870171 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870177 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870182 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870187 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870191 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870195 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870198 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870202 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870206 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870213 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870217 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870221 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870225 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870228 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870232 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870236 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870239 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870243 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870247 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870251 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 19:37:57.202407 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870254 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870257 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870263 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870267 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870271 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870275 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870279 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870283 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870289 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870293 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870297 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870300 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.870308 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870495 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870505 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 19:37:57.203264 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870510 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870514 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870518 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870522 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870526 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870530 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870536 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870540 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870543 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870547 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870551 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870555 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870559 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870562 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870566 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870569 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870574 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870577 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870581 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870585 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 19:37:57.203920 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870590 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870594 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870597 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870601 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870605 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870608 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870613 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870617 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870621 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870625 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870630 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870634 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870637 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870640 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870643 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870645 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870649 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870653 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870658 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870662 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 19:37:57.204791 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870666 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870670 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870674 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870678 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870684 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870688 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870692 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870696 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870700 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870704 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870707 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870711 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870717 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870721 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870725 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870731 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870736 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870740 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870745 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 19:37:57.412309 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870750 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870754 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870758 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870762 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870766 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870770 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870774 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870778 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870782 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870785 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870789 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870797 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870801 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870805 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870808 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870812 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870816 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870820 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870824 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870828 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 19:37:57.414081 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870831 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870835 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870840 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870843 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870848 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870853 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870857 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870861 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870865 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870868 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870872 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870876 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870880 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870885 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870890 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870897 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870901 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 19:37:57.414942 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:56.870905 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.870912 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.871065 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.873779 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.875509 2568 server.go:1019] "Starting client certificate rotation" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.875628 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.875662 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.878512 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.879928 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.893865 2568 log.go:25] "Validated CRI v1 runtime API" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.896580 2568 log.go:25] "Validated CRI v1 image API" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.897351 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 19:37:57.415812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.900956 2568 fs.go:135] Filesystem UUIDs: map[29dc067e-8e63-4db7-bc0c-6da0825b05fd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cc3f7d0b-81d8-461d-bcc1-b571f6a30311:/dev/nvme0n1p4] Jul 14 19:37:57.416661 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.900974 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 19:37:57.416876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.910460 2568 manager.go:216] Machine: {Timestamp:2026-07-14 19:37:56.907865099 +0000 UTC m=+0.381963850 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3120547 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22d188a9d466b5ebf11cd515f4e5b7 SystemUUID:ec22d188-a9d4-66b5-ebf1-1cd515f4e5b7 BootID:d48e46b6-6f87-4c0c-bc4c-73bc77d5130a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:13:d5:9f:93:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:13:d5:9f:93:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:4d:f6:31:f9:c4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 19:37:57.416876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.910624 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 19:37:57.416876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.910728 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 19:37:57.416876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.914450 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 19:37:57.416876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.914483 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-126.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 19:37:57.416876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.915279 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 19:37:57.416876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.915289 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 19:37:57.416876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.915297 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.915313 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.916613 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.917858 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.921519 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.921665 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.921693 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.921705 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.921725 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.921735 2568 kubelet.go:398] "Adding apiserver pod source" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.921745 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.922087 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.922102 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.925431 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.927307 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.927337 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928118 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928138 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928148 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928157 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928169 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928176 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928227 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928234 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928241 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928246 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928255 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.928267 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.929090 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.929098 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.930607 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.930675 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.930869 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-126.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.932721 2568 server.go:1268] "Started kubelet" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.932831 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.932881 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.933503 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.936965 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.937885 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 14 19:37:57.419138 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.942523 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.943341 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.943356 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.943432 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944151 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944154 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944174 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944257 2568 factory.go:55] Registering systemd factory Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944273 2568 factory.go:223] Registration of the systemd container factory successfully Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944281 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944291 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944541 2568 factory.go:153] Registering CRI-O factory Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944553 2568 factory.go:223] Registration of the crio container factory successfully Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944611 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944631 2568 factory.go:103] Registering Raw factory Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.944642 2568 manager.go:1196] Started watching for new ooms in manager Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.945017 2568 manager.go:349] Starting recovery of all containers Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.946445 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.946541 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.945620 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-126.ec2.internal.18c23f2fcd4b2604 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-126.ec2.internal,UID:ip-10-0-128-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-126.ec2.internal,},FirstTimestamp:2026-07-14 19:37:56.93240474 +0000 UTC m=+0.406503471,LastTimestamp:2026-07-14 19:37:56.93240474 +0000 UTC m=+0.406503471,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-126.ec2.internal,}" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.951457 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.956074 2568 manager.go:354] Recovery completed Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.960461 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.962712 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientMemory" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.962745 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.962757 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientPID" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.963285 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.963295 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.963312 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.963313 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.964101 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientMemory" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.964125 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 19:37:57.420206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.964136 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientPID" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.965225 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-126.ec2.internal.18c23f2fcf19d818 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-126.ec2.internal,UID:ip-10-0-128-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-126.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-126.ec2.internal,},FirstTimestamp:2026-07-14 19:37:56.96272796 +0000 UTC m=+0.436826695,LastTimestamp:2026-07-14 19:37:56.96272796 +0000 UTC m=+0.436826695,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-126.ec2.internal,}" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.965600 2568 policy_none.go:49] "None policy: Start" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.965634 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.965648 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.967831 2568 policy_none.go:47] "Start" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.973707 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-55qsw" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.975520 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-126.ec2.internal.18c23f2fcf1a344b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-126.ec2.internal,UID:ip-10-0-128-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-126.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-126.ec2.internal,},FirstTimestamp:2026-07-14 19:37:56.962751563 +0000 UTC m=+0.436850294,LastTimestamp:2026-07-14 19:37:56.962751563 +0000 UTC m=+0.436850294,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-126.ec2.internal,}" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:56.983294 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-55qsw" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:56.987158 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-126.ec2.internal.18c23f2fcf1a5c5f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-126.ec2.internal,UID:ip-10-0-128-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-126.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-126.ec2.internal,},FirstTimestamp:2026-07-14 19:37:56.962761823 +0000 UTC m=+0.436860554,LastTimestamp:2026-07-14 19:37:56.962761823 +0000 UTC m=+0.436860554,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-126.ec2.internal,}" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.016721 2568 manager.go:337] "Starting Device Plugin manager" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.016756 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.016771 2568 server.go:87] "Starting device plugin registration server" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.017044 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.017058 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.017184 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.017263 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.017272 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.017860 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.017908 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.474076 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.038894 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.038935 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.038957 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.038995 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.040906 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.117591 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.118670 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientMemory" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.118702 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.118713 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientPID" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.118748 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.129003 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.129030 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-126.ec2.internal\": node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.139255 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal"] Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.139318 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.140242 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientMemory" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.140271 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.140282 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientPID" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.142275 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.142595 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.142700 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.142723 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.143303 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientMemory" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.143326 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.143341 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientPID" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.143351 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientMemory" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.143368 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.143389 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientPID" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.145208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fb07ecdee4483866aa94f027039ff7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal\" (UID: \"8fb07ecdee4483866aa94f027039ff7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.145232 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fb07ecdee4483866aa94f027039ff7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal\" (UID: \"8fb07ecdee4483866aa94f027039ff7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.475155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.145249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/443a9d45f6d9f6d7163dde949a4b961a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.ec2.internal\" (UID: \"443a9d45f6d9f6d7163dde949a4b961a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.145574 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.145603 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.147697 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientMemory" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.147720 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.147729 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientPID" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.213702 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-126.ec2.internal\" not found" node="ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.218474 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-126.ec2.internal\" not found" node="ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.243223 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.245457 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fb07ecdee4483866aa94f027039ff7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal\" (UID: \"8fb07ecdee4483866aa94f027039ff7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.245483 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fb07ecdee4483866aa94f027039ff7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal\" (UID: \"8fb07ecdee4483866aa94f027039ff7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.245507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/443a9d45f6d9f6d7163dde949a4b961a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.ec2.internal\" (UID: \"443a9d45f6d9f6d7163dde949a4b961a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.245555 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fb07ecdee4483866aa94f027039ff7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal\" (UID: \"8fb07ecdee4483866aa94f027039ff7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.245563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fb07ecdee4483866aa94f027039ff7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal\" (UID: \"8fb07ecdee4483866aa94f027039ff7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.245555 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/443a9d45f6d9f6d7163dde949a4b961a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.ec2.internal\" (UID: \"443a9d45f6d9f6d7163dde949a4b961a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.343701 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.476693 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.444349 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.593995 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.516835 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.593995 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.521730 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" Jul 14 19:37:57.593995 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.544558 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.830909 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.644826 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:57.830909 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.747779 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.847881 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.874063 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.874249 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:57.947980 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.967228 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.973662 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientMemory" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.973694 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.973709 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeHasSufficientPID" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.989462 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 19:32:57 +0000 UTC" deadline="2028-04-29 18:06:04.203375698 +0000 UTC" Jul 14 19:37:58.007364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:57.989492 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15718h28m6.213887011s" Jul 14 19:37:58.163246 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:58.048717 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-126.ec2.internal\" not found" Jul 14 19:37:58.163246 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.103082 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 19:37:58.163246 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.144069 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" Jul 14 19:37:58.163246 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.162549 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 19:37:58.164684 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.164438 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" Jul 14 19:37:58.234836 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.207223 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 19:37:58.320019 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:58.253722 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443a9d45f6d9f6d7163dde949a4b961a.slice/crio-5e02bda6588498d484cbfb0d0a4aba806e90f701555d71abd4871928ae7bb5ad WatchSource:0}: Error finding container 5e02bda6588498d484cbfb0d0a4aba806e90f701555d71abd4871928ae7bb5ad: Status 404 returned error can't find the container with id 5e02bda6588498d484cbfb0d0a4aba806e90f701555d71abd4871928ae7bb5ad Jul 14 19:37:58.320019 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:58.259319 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb07ecdee4483866aa94f027039ff7d.slice/crio-9a38f1a0745f444d615099735dde41d8b44769657fbc08b3eb15d9a429ce3106 WatchSource:0}: Error finding container 9a38f1a0745f444d615099735dde41d8b44769657fbc08b3eb15d9a429ce3106: Status 404 returned error can't find the container with id 9a38f1a0745f444d615099735dde41d8b44769657fbc08b3eb15d9a429ce3106 Jul 14 19:37:58.320019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.263345 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 19:37:58.320019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.273428 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 19:37:58.667700 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.479034 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.923524 2568 apiserver.go:52] "Watching apiserver" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.931738 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.933297 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-rqk5f","kube-system/konnectivity-agent-krgkr","kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7","openshift-dns/node-resolver-9pmc8","openshift-image-registry/node-ca-xd4sq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal","openshift-multus/multus-additional-cni-plugins-58jks","openshift-cluster-node-tuning-operator/tuned-g6bpd","openshift-multus/multus-tj2xg","openshift-multus/network-metrics-daemon-vg8jr","openshift-network-diagnostics/network-check-target-jq9xp","openshift-network-operator/iptables-alerter-dshdc","openshift-ovn-kubernetes/ovnkube-node-h7wbn"] Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.936107 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tj2xg" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.938585 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.941129 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.943653 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.944023 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.946354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.955890 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956447 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-cni-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956473 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-cnibin\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956498 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-conf-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956517 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-kubelet\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-run-netns\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956550 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-var-lib-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956577 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-env-overrides\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.997435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-hostroot\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956648 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-node-log\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovnkube-script-lib\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956696 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956718 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-registration-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956742 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956769 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-k8s-cni-cncf-io\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956797 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5lfc\" (UniqueName: \"kubernetes.io/projected/5572ed36-c92e-474c-b738-f962ab34ae1a-kube-api-access-j5lfc\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956833 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bd55968-8301-4303-aa1a-f9853a39015d-tmp-dir\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956857 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-socket-dir-parent\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956884 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v8cv\" (UniqueName: \"kubernetes.io/projected/469094bf-3160-4036-a837-37e55eaa9323-kube-api-access-2v8cv\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956900 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:58.998898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-run-ovn-kubernetes\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956939 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a854c0d-8da0-4b27-9906-e4e0d622c98a-cni-binary-copy\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956955 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-netns\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956971 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9nc6\" (UniqueName: \"kubernetes.io/projected/1a854c0d-8da0-4b27-9906-e4e0d622c98a-kube-api-access-t9nc6\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.956985 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-systemd-units\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957000 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-ovn\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957016 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovnkube-config\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957031 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-cni-bin\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957058 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-multus-certs\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957086 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/469094bf-3160-4036-a837-37e55eaa9323-iptables-alerter-script\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957122 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-slash\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957176 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-etc-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957221 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-socket-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957255 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-device-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.001011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957275 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-system-cni-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-daemon-config\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957317 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-etc-kubernetes\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957359 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/469094bf-3160-4036-a837-37e55eaa9323-host-slash\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957381 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-cni-bin\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-cni-netd\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957455 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qzz7\" (UniqueName: \"kubernetes.io/projected/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-kube-api-access-2qzz7\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957486 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bd55968-8301-4303-aa1a-f9853a39015d-hosts-file\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957506 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-os-release\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-cni-multus\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-kubelet\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957588 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-systemd\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-log-socket\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957633 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovn-node-metrics-cert\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.002019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-sys-fs\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.957673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vftj4\" (UniqueName: \"kubernetes.io/projected/5bd55968-8301-4303-aa1a-f9853a39015d-kube-api-access-vftj4\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.961079 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.961278 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.961303 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.961580 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.961920 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.962113 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.962126 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.962390 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9qk8w\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.962512 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.962733 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.962750 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.962803 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xnpcv\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.962984 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.963058 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zvq7f\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.963070 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.963124 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.963200 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.963221 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rhdc7\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.963248 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.963350 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.963354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.965115 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.965320 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.965599 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.966495 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.967323 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.967613 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.967635 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 19:37:59.005255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.967690 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.967976 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9s2gl\"" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.968012 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-47hc6\"" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.968079 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5sk5g\"" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.968457 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.971237 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.971379 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.971482 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9f8vd\"" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.971245 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:58.971634 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.974193 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:58.974256 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:58.976624 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:37:59.007431 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:58.976690 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.046955 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.048146 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" event={"ID":"443a9d45f6d9f6d7163dde949a4b961a","Type":"ContainerStarted","Data":"5e02bda6588498d484cbfb0d0a4aba806e90f701555d71abd4871928ae7bb5ad"} Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.049645 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" event={"ID":"8fb07ecdee4483866aa94f027039ff7d","Type":"ContainerStarted","Data":"9a38f1a0745f444d615099735dde41d8b44769657fbc08b3eb15d9a429ce3106"} Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-cnibin\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-conf-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058207 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-kubelet\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058230 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-run-netns\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058254 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-var-lib-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058279 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-cnibin\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-env-overrides\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058322 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-os-release\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-run-netns\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058285 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-conf-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058351 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.078857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058354 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-var-lib-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058380 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovnkube-script-lib\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058347 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-kubelet\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058463 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058514 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058524 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058545 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d84bb427-bbba-4184-9a51-db0123e2137c-etc-tuned\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058577 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-k8s-cni-cncf-io\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bd55968-8301-4303-aa1a-f9853a39015d-tmp-dir\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058649 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-k8s-cni-cncf-io\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058670 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysctl-conf\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-systemd\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.079807 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058718 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-sys\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058694 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqqnz\" (UniqueName: \"kubernetes.io/projected/d84bb427-bbba-4184-9a51-db0123e2137c-kube-api-access-kqqnz\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-run-ovn-kubernetes\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-host\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d84bb427-bbba-4184-9a51-db0123e2137c-tmp\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-modprobe-d\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058916 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysconfig\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058920 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-run-ovn-kubernetes\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.058948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a854c0d-8da0-4b27-9906-e4e0d622c98a-cni-binary-copy\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9nc6\" (UniqueName: \"kubernetes.io/projected/1a854c0d-8da0-4b27-9906-e4e0d622c98a-kube-api-access-t9nc6\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059534 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-systemd-units\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059582 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a854c0d-8da0-4b27-9906-e4e0d622c98a-cni-binary-copy\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059593 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-systemd-units\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081213 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059621 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-ovn\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovnkube-config\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059684 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-device-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059710 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-cni-bin\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059741 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-multus-certs\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059657 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-ovn\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/469094bf-3160-4036-a837-37e55eaa9323-iptables-alerter-script\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-slash\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059817 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-var-lib-kubelet\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059842 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ba863c7-4529-497a-a1b0-a4a178f229b7-konnectivity-ca\") pod \"konnectivity-agent-krgkr\" (UID: \"7ba863c7-4529-497a-a1b0-a4a178f229b7\") " pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-system-cni-dir\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8v4f\" (UniqueName: \"kubernetes.io/projected/33d370ea-b86b-4e86-a173-0babca5f40e0-kube-api-access-k8v4f\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059924 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-system-cni-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-etc-kubernetes\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.081954 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059973 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/469094bf-3160-4036-a837-37e55eaa9323-host-slash\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.059996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-cni-bin\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060024 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060069 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-cni-netd\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-cni-bin\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060153 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-multus-certs\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060203 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-device-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060298 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060727 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nqncm\"" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/469094bf-3160-4036-a837-37e55eaa9323-iptables-alerter-script\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060984 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-slash\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-system-cni-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.060020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-cni-netd\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061166 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-etc-kubernetes\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061182 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bd55968-8301-4303-aa1a-f9853a39015d-hosts-file\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061206 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/469094bf-3160-4036-a837-37e55eaa9323-host-slash\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c889a0ed-dd2e-4229-9fca-3417dfdd901a-host\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.083140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061247 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c889a0ed-dd2e-4229-9fca-3417dfdd901a-serviceca\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061251 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-host-cni-bin\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-os-release\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061301 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-cni-multus\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-systemd\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061345 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bd55968-8301-4303-aa1a-f9853a39015d-hosts-file\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-log-socket\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061383 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovn-node-metrics-cert\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061415 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-os-release\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061437 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061479 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-cni-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061483 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-log-socket\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061504 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-hostroot\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061524 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-cni-multus\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061566 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-systemd\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.084108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061750 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-node-log\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061775 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-registration-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061794 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-kubernetes\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061795 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-cni-dir\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061809 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysctl-d\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061825 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-run\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061842 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5lfc\" (UniqueName: \"kubernetes.io/projected/5572ed36-c92e-474c-b738-f962ab34ae1a-kube-api-access-j5lfc\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061852 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-hostroot\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061885 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qndbc\" (UniqueName: \"kubernetes.io/projected/c889a0ed-dd2e-4229-9fca-3417dfdd901a-kube-api-access-qndbc\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061903 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061906 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-node-log\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2rsb\" (UniqueName: \"kubernetes.io/projected/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-kube-api-access-d2rsb\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061966 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-registration-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.061995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-socket-dir-parent\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.085289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062024 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v8cv\" (UniqueName: \"kubernetes.io/projected/469094bf-3160-4036-a837-37e55eaa9323-kube-api-access-2v8cv\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062076 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df43b1ab-25e2-4928-80e8-91d3c6e80a59-dbus\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-netns\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-socket-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062170 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-sys-fs\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-etc-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062232 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062270 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-daemon-config\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062298 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qzz7\" (UniqueName: \"kubernetes.io/projected/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-kube-api-access-2qzz7\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-cnibin\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df43b1ab-25e2-4928-80e8-91d3c6e80a59-kubelet-config\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062357 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-run-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062426 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-etc-openvswitch\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.091481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062430 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062302 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-socket-dir-parent\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062505 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-kubelet\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vftj4\" (UniqueName: \"kubernetes.io/projected/5bd55968-8301-4303-aa1a-f9853a39015d-kube-api-access-vftj4\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062537 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-run-netns\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-lib-modules\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062333 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-sys-fs\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062712 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5572ed36-c92e-474c-b738-f962ab34ae1a-socket-dir\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.062763 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a854c0d-8da0-4b27-9906-e4e0d622c98a-host-var-lib-kubelet\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.066131 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovnkube-script-lib\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.066375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ba863c7-4529-497a-a1b0-a4a178f229b7-agent-certs\") pod \"konnectivity-agent-krgkr\" (UID: \"7ba863c7-4529-497a-a1b0-a4a178f229b7\") " pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.066400 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.066419 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.066511 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a854c0d-8da0-4b27-9906-e4e0d622c98a-multus-daemon-config\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.092198 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.066873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-env-overrides\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.092838 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.067234 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bd55968-8301-4303-aa1a-f9853a39015d-tmp-dir\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.092838 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.069614 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovnkube-config\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.092838 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.072558 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 19:37:59.092838 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.087002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-ovn-node-metrics-cert\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.135959 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.135897 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 19:37:59.136126 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.136114 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tfbxx" Jul 14 19:37:59.136712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.136682 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5lfc\" (UniqueName: \"kubernetes.io/projected/5572ed36-c92e-474c-b738-f962ab34ae1a-kube-api-access-j5lfc\") pod \"aws-ebs-csi-driver-node-d6xt7\" (UID: \"5572ed36-c92e-474c-b738-f962ab34ae1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.136854 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.136802 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v8cv\" (UniqueName: \"kubernetes.io/projected/469094bf-3160-4036-a837-37e55eaa9323-kube-api-access-2v8cv\") pod \"iptables-alerter-dshdc\" (UID: \"469094bf-3160-4036-a837-37e55eaa9323\") " pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.138582 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.138501 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9nc6\" (UniqueName: \"kubernetes.io/projected/1a854c0d-8da0-4b27-9906-e4e0d622c98a-kube-api-access-t9nc6\") pod \"multus-tj2xg\" (UID: \"1a854c0d-8da0-4b27-9906-e4e0d622c98a\") " pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.151723 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.151693 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vftj4\" (UniqueName: \"kubernetes.io/projected/5bd55968-8301-4303-aa1a-f9853a39015d-kube-api-access-vftj4\") pod \"node-resolver-9pmc8\" (UID: \"5bd55968-8301-4303-aa1a-f9853a39015d\") " pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.152098 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.152072 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qzz7\" (UniqueName: \"kubernetes.io/projected/d07927f8-3b5c-4f1d-a3e8-ad78915b6c10-kube-api-access-2qzz7\") pod \"ovnkube-node-h7wbn\" (UID: \"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.152245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.152229 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" Jul 14 19:37:59.165950 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.165755 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9pmc8" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167328 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-lib-modules\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167380 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ba863c7-4529-497a-a1b0-a4a178f229b7-agent-certs\") pod \"konnectivity-agent-krgkr\" (UID: \"7ba863c7-4529-497a-a1b0-a4a178f229b7\") " pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167426 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167504 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-os-release\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167530 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d84bb427-bbba-4184-9a51-db0123e2137c-etc-tuned\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167599 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysctl-conf\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167622 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-systemd\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-sys\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167668 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqqnz\" (UniqueName: \"kubernetes.io/projected/d84bb427-bbba-4184-9a51-db0123e2137c-kube-api-access-kqqnz\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.167697 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167692 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-host\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d84bb427-bbba-4184-9a51-db0123e2137c-tmp\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167732 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-os-release\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167737 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-modprobe-d\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysconfig\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-var-lib-kubelet\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167863 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-modprobe-d\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ba863c7-4529-497a-a1b0-a4a178f229b7-konnectivity-ca\") pod \"konnectivity-agent-krgkr\" (UID: \"7ba863c7-4529-497a-a1b0-a4a178f229b7\") " pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167909 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-system-cni-dir\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167936 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8v4f\" (UniqueName: \"kubernetes.io/projected/33d370ea-b86b-4e86-a173-0babca5f40e0-kube-api-access-k8v4f\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167967 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c889a0ed-dd2e-4229-9fca-3417dfdd901a-host\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.167989 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c889a0ed-dd2e-4229-9fca-3417dfdd901a-serviceca\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168033 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168087 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-kubernetes\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.168345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysctl-d\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-run\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168170 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qndbc\" (UniqueName: \"kubernetes.io/projected/c889a0ed-dd2e-4229-9fca-3417dfdd901a-kube-api-access-qndbc\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168223 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rsb\" (UniqueName: \"kubernetes.io/projected/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-kube-api-access-d2rsb\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168254 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df43b1ab-25e2-4928-80e8-91d3c6e80a59-dbus\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168317 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-cnibin\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df43b1ab-25e2-4928-80e8-91d3c6e80a59-kubelet-config\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168374 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.168550 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.168556 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.169175 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.168649 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:37:59.668610747 +0000 UTC m=+3.142709489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:37:59.169818 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.169216 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-lib-modules\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.171559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.170238 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.171559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.170741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/33d370ea-b86b-4e86-a173-0babca5f40e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.171559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.170802 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-system-cni-dir\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.171559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.170896 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ba863c7-4529-497a-a1b0-a4a178f229b7-konnectivity-ca\") pod \"konnectivity-agent-krgkr\" (UID: \"7ba863c7-4529-497a-a1b0-a4a178f229b7\") " pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:37:59.171559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.171067 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c889a0ed-dd2e-4229-9fca-3417dfdd901a-host\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.171559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.171151 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysconfig\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.171559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.171416 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-var-lib-kubelet\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172101 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.171650 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c889a0ed-dd2e-4229-9fca-3417dfdd901a-serviceca\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.172101 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.171969 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-sys\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172101 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172095 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysctl-conf\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172354 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172104 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d84bb427-bbba-4184-9a51-db0123e2137c-etc-tuned\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172354 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172174 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-systemd\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172354 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.172192 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 19:37:59.172354 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.172252 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret podName:df43b1ab-25e2-4928-80e8-91d3c6e80a59 nodeName:}" failed. No retries permitted until 2026-07-14 19:37:59.67223519 +0000 UTC m=+3.146333919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret") pod "global-pull-secret-syncer-rqk5f" (UID: "df43b1ab-25e2-4928-80e8-91d3c6e80a59") : object "kube-system"/"original-pull-secret" not registered Jul 14 19:37:59.172616 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172424 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-kubernetes\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172616 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172536 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-host\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172616 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172557 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-etc-sysctl-d\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172853 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172616 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d84bb427-bbba-4184-9a51-db0123e2137c-run\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.172853 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172682 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.172853 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172694 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df43b1ab-25e2-4928-80e8-91d3c6e80a59-dbus\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.172853 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172734 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df43b1ab-25e2-4928-80e8-91d3c6e80a59-kubelet-config\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.172853 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.172746 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33d370ea-b86b-4e86-a173-0babca5f40e0-cnibin\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.174947 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.173769 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ba863c7-4529-497a-a1b0-a4a178f229b7-agent-certs\") pod \"konnectivity-agent-krgkr\" (UID: \"7ba863c7-4529-497a-a1b0-a4a178f229b7\") " pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:37:59.176292 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.176258 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d84bb427-bbba-4184-9a51-db0123e2137c-tmp\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.180344 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:59.180313 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd55968_8301_4303_aa1a_f9853a39015d.slice/crio-636be83e07a285406fbd1a630896067f1e2510ab64c5399a52e83760cb3a6f68 WatchSource:0}: Error finding container 636be83e07a285406fbd1a630896067f1e2510ab64c5399a52e83760cb3a6f68: Status 404 returned error can't find the container with id 636be83e07a285406fbd1a630896067f1e2510ab64c5399a52e83760cb3a6f68 Jul 14 19:37:59.181048 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.181023 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 19:37:59.181161 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.181053 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 19:37:59.181161 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.181069 2568 projected.go:196] Error preparing data for projected volume kube-api-access-vspqz for pod openshift-network-diagnostics/network-check-target-jq9xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:37:59.181161 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.181141 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz podName:ec294756-8328-40b8-b4b1-36ca8c695ca5 nodeName:}" failed. No retries permitted until 2026-07-14 19:37:59.681119364 +0000 UTC m=+3.155218109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vspqz" (UniqueName: "kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz") pod "network-check-target-jq9xp" (UID: "ec294756-8328-40b8-b4b1-36ca8c695ca5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:37:59.184793 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.184729 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qndbc\" (UniqueName: \"kubernetes.io/projected/c889a0ed-dd2e-4229-9fca-3417dfdd901a-kube-api-access-qndbc\") pod \"node-ca-xd4sq\" (UID: \"c889a0ed-dd2e-4229-9fca-3417dfdd901a\") " pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.184901 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.184835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8v4f\" (UniqueName: \"kubernetes.io/projected/33d370ea-b86b-4e86-a173-0babca5f40e0-kube-api-access-k8v4f\") pod \"multus-additional-cni-plugins-58jks\" (UID: \"33d370ea-b86b-4e86-a173-0babca5f40e0\") " pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.189774 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.189715 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqqnz\" (UniqueName: \"kubernetes.io/projected/d84bb427-bbba-4184-9a51-db0123e2137c-kube-api-access-kqqnz\") pod \"tuned-g6bpd\" (UID: \"d84bb427-bbba-4184-9a51-db0123e2137c\") " pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.196561 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.196528 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rsb\" (UniqueName: \"kubernetes.io/projected/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-kube-api-access-d2rsb\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:37:59.197998 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.197974 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-58jks" Jul 14 19:37:59.205361 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.205335 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:37:59.213837 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:59.213800 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d370ea_b86b_4e86_a173_0babca5f40e0.slice/crio-f5223e3287d5ea330fe755c39392133f7426460d0245e673a8b087dda005f9a1 WatchSource:0}: Error finding container f5223e3287d5ea330fe755c39392133f7426460d0245e673a8b087dda005f9a1: Status 404 returned error can't find the container with id f5223e3287d5ea330fe755c39392133f7426460d0245e673a8b087dda005f9a1 Jul 14 19:37:59.216628 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.216279 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 19:37:59.217446 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:59.217376 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba863c7_4529_497a_a1b0_a4a178f229b7.slice/crio-e0378fb6bac2cae0a62cd806be2fdaf1c248dfdaa2e943cf536e2f86d5921919 WatchSource:0}: Error finding container e0378fb6bac2cae0a62cd806be2fdaf1c248dfdaa2e943cf536e2f86d5921919: Status 404 returned error can't find the container with id e0378fb6bac2cae0a62cd806be2fdaf1c248dfdaa2e943cf536e2f86d5921919 Jul 14 19:37:59.248909 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.248873 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tj2xg" Jul 14 19:37:59.261627 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.261591 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dshdc" Jul 14 19:37:59.267472 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:59.267416 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a854c0d_8da0_4b27_9906_e4e0d622c98a.slice/crio-aae9786d722067410e8295306abf6b69751a7be96dc1a6807362b35eb2fd9079 WatchSource:0}: Error finding container aae9786d722067410e8295306abf6b69751a7be96dc1a6807362b35eb2fd9079: Status 404 returned error can't find the container with id aae9786d722067410e8295306abf6b69751a7be96dc1a6807362b35eb2fd9079 Jul 14 19:37:59.275511 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:37:59.275476 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469094bf_3160_4036_a837_37e55eaa9323.slice/crio-9ed7410399aa75b558c40f8ade018156ccbe0464e5d5d95aa1034bf46f5a6aea WatchSource:0}: Error finding container 9ed7410399aa75b558c40f8ade018156ccbe0464e5d5d95aa1034bf46f5a6aea: Status 404 returned error can't find the container with id 9ed7410399aa75b558c40f8ade018156ccbe0464e5d5d95aa1034bf46f5a6aea Jul 14 19:37:59.370189 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.370092 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:37:59.478640 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.478597 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xd4sq" Jul 14 19:37:59.488231 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.487992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" Jul 14 19:37:59.672986 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.672879 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:37:59.672986 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.672958 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:37:59.673240 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.673066 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 19:37:59.673240 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.673103 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:37:59.673240 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.673162 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret podName:df43b1ab-25e2-4928-80e8-91d3c6e80a59 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:00.673140068 +0000 UTC m=+4.147238810 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret") pod "global-pull-secret-syncer-rqk5f" (UID: "df43b1ab-25e2-4928-80e8-91d3c6e80a59") : object "kube-system"/"original-pull-secret" not registered Jul 14 19:37:59.673240 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.673184 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:00.673173821 +0000 UTC m=+4.147272544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:37:59.774158 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:37:59.774112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:37:59.774354 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.774317 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 19:37:59.774354 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.774349 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 19:37:59.774542 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.774363 2568 projected.go:196] Error preparing data for projected volume kube-api-access-vspqz for pod openshift-network-diagnostics/network-check-target-jq9xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:37:59.774542 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:37:59.774445 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz podName:ec294756-8328-40b8-b4b1-36ca8c695ca5 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:00.774424756 +0000 UTC m=+4.248523483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vspqz" (UniqueName: "kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz") pod "network-check-target-jq9xp" (UID: "ec294756-8328-40b8-b4b1-36ca8c695ca5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:00.040103 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.040018 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:00.040601 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.040138 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:00.052056 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.052013 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9pmc8" event={"ID":"5bd55968-8301-4303-aa1a-f9853a39015d","Type":"ContainerStarted","Data":"636be83e07a285406fbd1a630896067f1e2510ab64c5399a52e83760cb3a6f68"} Jul 14 19:38:00.053335 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.053296 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" event={"ID":"5572ed36-c92e-474c-b738-f962ab34ae1a","Type":"ContainerStarted","Data":"7f3a1c2cd22c9adf98df8d108fb8ba91feb4954a94fa664c57fd60860214ead7"} Jul 14 19:38:00.054549 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.054513 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dshdc" event={"ID":"469094bf-3160-4036-a837-37e55eaa9323","Type":"ContainerStarted","Data":"9ed7410399aa75b558c40f8ade018156ccbe0464e5d5d95aa1034bf46f5a6aea"} Jul 14 19:38:00.055749 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.055712 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tj2xg" event={"ID":"1a854c0d-8da0-4b27-9906-e4e0d622c98a","Type":"ContainerStarted","Data":"aae9786d722067410e8295306abf6b69751a7be96dc1a6807362b35eb2fd9079"} Jul 14 19:38:00.056818 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.056797 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-krgkr" event={"ID":"7ba863c7-4529-497a-a1b0-a4a178f229b7","Type":"ContainerStarted","Data":"e0378fb6bac2cae0a62cd806be2fdaf1c248dfdaa2e943cf536e2f86d5921919"} Jul 14 19:38:00.057921 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.057897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerStarted","Data":"f5223e3287d5ea330fe755c39392133f7426460d0245e673a8b087dda005f9a1"} Jul 14 19:38:00.142778 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.142732 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 19:32:59 +0000 UTC" deadline="2028-02-17 06:19:55.683043732 +0000 UTC" Jul 14 19:38:00.142778 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.142776 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13978h41m55.540273179s" Jul 14 19:38:00.681530 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.681484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:00.681757 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.681553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:00.681757 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.681717 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:00.681894 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.681790 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:02.68176761 +0000 UTC m=+6.155866347 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:00.681894 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.681717 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:00.681894 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.681881 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret podName:df43b1ab-25e2-4928-80e8-91d3c6e80a59 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:02.681859584 +0000 UTC m=+6.155958314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret") pod "global-pull-secret-syncer-rqk5f" (UID: "df43b1ab-25e2-4928-80e8-91d3c6e80a59") : object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:00.782245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:00.782201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:00.782444 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.782390 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 19:38:00.782444 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.782416 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 19:38:00.782444 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.782448 2568 projected.go:196] Error preparing data for projected volume kube-api-access-vspqz for pod openshift-network-diagnostics/network-check-target-jq9xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:00.782640 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:00.782519 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz podName:ec294756-8328-40b8-b4b1-36ca8c695ca5 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:02.782493529 +0000 UTC m=+6.256592274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vspqz" (UniqueName: "kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz") pod "network-check-target-jq9xp" (UID: "ec294756-8328-40b8-b4b1-36ca8c695ca5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:00.823421 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:38:00.823372 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07927f8_3b5c_4f1d_a3e8_ad78915b6c10.slice/crio-bf62c6cb38d27559abebdc2b4846dfa9c750cd4d5f8518d50df1ea04b39cecf8 WatchSource:0}: Error finding container bf62c6cb38d27559abebdc2b4846dfa9c750cd4d5f8518d50df1ea04b39cecf8: Status 404 returned error can't find the container with id bf62c6cb38d27559abebdc2b4846dfa9c750cd4d5f8518d50df1ea04b39cecf8 Jul 14 19:38:00.832930 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:38:00.832895 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd84bb427_bbba_4184_9a51_db0123e2137c.slice/crio-6a408413f6831ae7c44b3c89387a81a7dcb6d8b7f34dccc0f7ce229005d09330 WatchSource:0}: Error finding container 6a408413f6831ae7c44b3c89387a81a7dcb6d8b7f34dccc0f7ce229005d09330: Status 404 returned error can't find the container with id 6a408413f6831ae7c44b3c89387a81a7dcb6d8b7f34dccc0f7ce229005d09330 Jul 14 19:38:01.044762 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.044727 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:01.045371 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:01.044867 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:01.045371 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.045313 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:01.045533 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:01.045435 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:01.066101 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.066060 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"bf62c6cb38d27559abebdc2b4846dfa9c750cd4d5f8518d50df1ea04b39cecf8"} Jul 14 19:38:01.072067 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.072027 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" event={"ID":"443a9d45f6d9f6d7163dde949a4b961a","Type":"ContainerStarted","Data":"f18b2c4b2872e9f04c857205f4e534a1bda375de122ed0b46baa4209c5629d08"} Jul 14 19:38:01.078702 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.078655 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" event={"ID":"d84bb427-bbba-4184-9a51-db0123e2137c","Type":"ContainerStarted","Data":"6a408413f6831ae7c44b3c89387a81a7dcb6d8b7f34dccc0f7ce229005d09330"} Jul 14 19:38:01.081473 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.081415 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xd4sq" event={"ID":"c889a0ed-dd2e-4229-9fca-3417dfdd901a","Type":"ContainerStarted","Data":"c19facfe60574280856c1562dba128268f52224b8c6116d43032eaddf275a93a"} Jul 14 19:38:01.092426 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.092332 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.ec2.internal" podStartSLOduration=3.092309849 podStartE2EDuration="3.092309849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:01.092246516 +0000 UTC m=+4.566345262" watchObservedRunningTime="2026-07-14 19:38:01.092309849 +0000 UTC m=+4.566408594" Jul 14 19:38:01.143657 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.143569 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 19:32:59 +0000 UTC" deadline="2028-01-24 18:31:22.038232189 +0000 UTC" Jul 14 19:38:01.143657 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:01.143643 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13414h53m20.89459384s" Jul 14 19:38:02.039608 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:02.039518 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:02.039800 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.039669 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:02.090206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:02.089433 2568 generic.go:356] "Generic (PLEG): container finished" podID="8fb07ecdee4483866aa94f027039ff7d" containerID="1024780f17b856183bac584417cd3ca0a1bee31de06006b7a5ccf343cd14ef69" exitCode=0 Jul 14 19:38:02.090206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:02.089669 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" event={"ID":"8fb07ecdee4483866aa94f027039ff7d","Type":"ContainerDied","Data":"1024780f17b856183bac584417cd3ca0a1bee31de06006b7a5ccf343cd14ef69"} Jul 14 19:38:02.701851 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:02.701805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:02.702072 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:02.701919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:02.702135 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.702074 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:02.702188 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.702141 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret podName:df43b1ab-25e2-4928-80e8-91d3c6e80a59 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:06.702120592 +0000 UTC m=+10.176219336 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret") pod "global-pull-secret-syncer-rqk5f" (UID: "df43b1ab-25e2-4928-80e8-91d3c6e80a59") : object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:02.702602 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.702575 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:02.702737 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.702640 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:06.702622213 +0000 UTC m=+10.176720936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:02.803450 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:02.803390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:02.803650 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.803605 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 19:38:02.803650 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.803630 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 19:38:02.803650 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.803643 2568 projected.go:196] Error preparing data for projected volume kube-api-access-vspqz for pod openshift-network-diagnostics/network-check-target-jq9xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:02.803827 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:02.803703 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz podName:ec294756-8328-40b8-b4b1-36ca8c695ca5 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:06.803684191 +0000 UTC m=+10.277782930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vspqz" (UniqueName: "kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz") pod "network-check-target-jq9xp" (UID: "ec294756-8328-40b8-b4b1-36ca8c695ca5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:03.041498 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:03.041320 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:03.041662 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:03.041502 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:03.041662 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:03.041572 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:03.041739 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:03.041686 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:04.039644 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:04.039609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:04.040150 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:04.039751 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:05.039557 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:05.039518 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:05.039747 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:05.039566 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:05.039747 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:05.039679 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:05.040108 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:05.039870 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:06.039798 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:06.039713 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:06.040243 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.039854 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:06.738955 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:06.738909 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:06.739153 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:06.738971 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:06.739153 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.739091 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:06.739153 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.739154 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:14.739135678 +0000 UTC m=+18.213234404 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:06.739660 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.739569 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:06.739660 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.739619 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret podName:df43b1ab-25e2-4928-80e8-91d3c6e80a59 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:14.739604828 +0000 UTC m=+18.213703552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret") pod "global-pull-secret-syncer-rqk5f" (UID: "df43b1ab-25e2-4928-80e8-91d3c6e80a59") : object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:06.840627 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:06.839926 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:06.840627 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.840154 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 19:38:06.840627 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.840176 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 19:38:06.840627 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.840189 2568 projected.go:196] Error preparing data for projected volume kube-api-access-vspqz for pod openshift-network-diagnostics/network-check-target-jq9xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:06.840627 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:06.840251 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz podName:ec294756-8328-40b8-b4b1-36ca8c695ca5 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:14.840232074 +0000 UTC m=+18.314330806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vspqz" (UniqueName: "kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz") pod "network-check-target-jq9xp" (UID: "ec294756-8328-40b8-b4b1-36ca8c695ca5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:07.041082 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:07.041002 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:07.041565 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:07.041143 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:07.041565 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:07.041356 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:07.041565 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:07.041514 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:08.040197 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:08.039989 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:08.040197 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:08.040135 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:09.039408 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:09.039202 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:09.039408 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:09.039361 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:09.039909 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:09.039786 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:09.039947 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:09.039897 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:10.039847 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:10.039800 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:10.040260 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:10.039946 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:11.039187 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:11.039146 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:11.039364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:11.039146 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:11.039364 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:11.039295 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:11.039364 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:11.039327 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:12.039813 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:12.039774 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:12.040273 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:12.039892 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:13.039764 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:13.039725 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:13.039950 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:13.039877 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:13.039950 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:13.039935 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:13.040402 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:13.040060 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:14.040090 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:14.040051 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:14.040598 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.040194 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:14.804654 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:14.804609 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:14.804654 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:14.804659 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:14.804878 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.804788 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:14.804878 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.804791 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:14.804878 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.804843 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:30.804829097 +0000 UTC m=+34.278927815 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:14.804878 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.804860 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret podName:df43b1ab-25e2-4928-80e8-91d3c6e80a59 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:30.804851295 +0000 UTC m=+34.278950015 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret") pod "global-pull-secret-syncer-rqk5f" (UID: "df43b1ab-25e2-4928-80e8-91d3c6e80a59") : object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:14.905673 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:14.905637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:14.905854 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.905807 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 19:38:14.905854 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.905832 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 19:38:14.905854 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.905843 2568 projected.go:196] Error preparing data for projected volume kube-api-access-vspqz for pod openshift-network-diagnostics/network-check-target-jq9xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:14.905971 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:14.905907 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz podName:ec294756-8328-40b8-b4b1-36ca8c695ca5 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:30.905888626 +0000 UTC m=+34.379987363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vspqz" (UniqueName: "kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz") pod "network-check-target-jq9xp" (UID: "ec294756-8328-40b8-b4b1-36ca8c695ca5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:15.039967 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:15.039936 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:15.040145 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:15.039942 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:15.040145 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:15.040060 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:15.040571 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:15.040146 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:16.040046 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:16.039956 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:16.040234 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:16.040099 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:17.040278 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:17.040234 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:17.040746 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:17.040362 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:17.040746 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:17.040414 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:17.040746 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:17.040572 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:18.039199 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:18.039162 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:18.039371 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:18.039305 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:19.039311 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:19.039260 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:19.039875 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:19.039269 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:19.039875 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:19.039410 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:19.039875 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:19.039510 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:20.040364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.040002 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:20.041088 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:20.040413 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:20.129928 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.129884 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-krgkr" event={"ID":"7ba863c7-4529-497a-a1b0-a4a178f229b7","Type":"ContainerStarted","Data":"cec42a793467370d745de0b4461a13dafd30f7b863a8fc4c3adce203f9852458"} Jul 14 19:38:20.131584 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.131552 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9pmc8" event={"ID":"5bd55968-8301-4303-aa1a-f9853a39015d","Type":"ContainerStarted","Data":"fc3976f8e3da84dfa3e1ab6f047f4f8df39c8b801e09f7ec0bec09e88f06e7e8"} Jul 14 19:38:20.134712 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.134684 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:38:20.135091 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.135069 2568 generic.go:356] "Generic (PLEG): container finished" podID="d07927f8-3b5c-4f1d-a3e8-ad78915b6c10" containerID="673939624a210407011c67ddae2ef0d3a1428caae8a6b9573fc8cdb735c856b5" exitCode=1 Jul 14 19:38:20.135212 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.135138 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"b9b12d210f80acf3c966680b758623c4d2e7ef929beb98a676c0c3e655d9f802"} Jul 14 19:38:20.135274 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.135227 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"dbbe094193e00f2fad2b2939f5204ec6bdc978e120b5fb9512df596fd62075a3"} Jul 14 19:38:20.135328 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.135280 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"4b92c90b3c9d2caf6453a52eed2be236318a0cbb62987d0196191b2f7986e65d"} Jul 14 19:38:20.135401 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.135331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"b7c4ef1cd58e15376747aa2090c9d8874f1097419ac0da22f2fd9a425e4debf0"} Jul 14 19:38:20.135461 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.135401 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerDied","Data":"673939624a210407011c67ddae2ef0d3a1428caae8a6b9573fc8cdb735c856b5"} Jul 14 19:38:20.135461 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.135455 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"087e14acfc04310b1d12056d3217ab81266255a49a00c5a2ad6531ef48049f46"} Jul 14 19:38:20.137221 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.137197 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" event={"ID":"8fb07ecdee4483866aa94f027039ff7d","Type":"ContainerStarted","Data":"9ce8db28b57a86b7f253348cc3357334e815c71bba158fe1a8483616dafcb0e2"} Jul 14 19:38:20.138857 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.138829 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tj2xg" event={"ID":"1a854c0d-8da0-4b27-9906-e4e0d622c98a","Type":"ContainerStarted","Data":"698c0cd4fb918513b8de39e5cea3028b9cc0e75543b310f1f60e7d8881f9d46c"} Jul 14 19:38:20.140333 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.140296 2568 generic.go:356] "Generic (PLEG): container finished" podID="33d370ea-b86b-4e86-a173-0babca5f40e0" containerID="4b91d4e0bdf73f5363ac84dd7dbb1dedee7669063b207f66863d3da08785f4ba" exitCode=0 Jul 14 19:38:20.140448 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.140395 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerDied","Data":"4b91d4e0bdf73f5363ac84dd7dbb1dedee7669063b207f66863d3da08785f4ba"} Jul 14 19:38:20.142172 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.142146 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" event={"ID":"5572ed36-c92e-474c-b738-f962ab34ae1a","Type":"ContainerStarted","Data":"b102292c93379de8b2c1576c4a214ea4050904edb9da641cf91973c682cafa9f"} Jul 14 19:38:20.144023 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.143971 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" event={"ID":"d84bb427-bbba-4184-9a51-db0123e2137c","Type":"ContainerStarted","Data":"7825f52980bafd1455082593c79cd0fac65beaa9e9ea4135ce23c60ce7f35486"} Jul 14 19:38:20.145570 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.145542 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xd4sq" event={"ID":"c889a0ed-dd2e-4229-9fca-3417dfdd901a","Type":"ContainerStarted","Data":"2d37365a7d644d45e86eb281175e453578d77df4f19c4044962c345606bc2e58"} Jul 14 19:38:20.149249 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.149193 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-krgkr" podStartSLOduration=3.195780629 podStartE2EDuration="23.14916526s" totalImagesPullingTime="19.953384631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:20.148442309 +0000 UTC m=+23.622541052" watchObservedRunningTime="2026-07-14 19:38:20.14916526 +0000 UTC m=+23.623264001" Jul 14 19:38:20.178511 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.178442 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.ec2.internal" podStartSLOduration=22.178425292 podStartE2EDuration="22.178425292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:20.178414986 +0000 UTC m=+23.652513728" watchObservedRunningTime="2026-07-14 19:38:20.178425292 +0000 UTC m=+23.652524035" Jul 14 19:38:20.201471 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.201412 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tj2xg" podStartSLOduration=3.213589544 podStartE2EDuration="23.201393656s" totalImagesPullingTime="19.987804112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:20.201273143 +0000 UTC m=+23.675371886" watchObservedRunningTime="2026-07-14 19:38:20.201393656 +0000 UTC m=+23.675492387" Jul 14 19:38:20.240170 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.240114 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9pmc8" podStartSLOduration=3.260017457 podStartE2EDuration="23.240097073s" totalImagesPullingTime="19.980079616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:20.220254083 +0000 UTC m=+23.694352837" watchObservedRunningTime="2026-07-14 19:38:20.240097073 +0000 UTC m=+23.714195863" Jul 14 19:38:20.240341 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.240319 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g6bpd" podStartSLOduration=4.878759321 podStartE2EDuration="23.240315004s" totalImagesPullingTime="18.361555683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:20.239744281 +0000 UTC m=+23.713843022" watchObservedRunningTime="2026-07-14 19:38:20.240315004 +0000 UTC m=+23.714413744" Jul 14 19:38:20.284906 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.284861 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xd4sq" podStartSLOduration=4.951660906 podStartE2EDuration="23.284844536s" totalImagesPullingTime="18.33318363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:20.284826431 +0000 UTC m=+23.758925171" watchObservedRunningTime="2026-07-14 19:38:20.284844536 +0000 UTC m=+23.758943276" Jul 14 19:38:20.826983 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:20.826943 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 19:38:21.029597 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:21.029431 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T19:38:20.826975002Z","UUID":"28e82de8-0104-4b70-9d3b-1fab3b5e7b91","Handler":null,"Name":"","Endpoint":""} Jul 14 19:38:21.031250 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:21.031227 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 19:38:21.031402 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:21.031274 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 19:38:21.039307 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:21.039281 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:21.039483 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:21.039283 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:21.039483 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:21.039440 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:21.039603 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:21.039492 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:21.150105 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:21.150050 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" event={"ID":"5572ed36-c92e-474c-b738-f962ab34ae1a","Type":"ContainerStarted","Data":"56c78271f87a1d8b098424e5482595106788c251cf09428bd2d5898a74150339"} Jul 14 19:38:21.151853 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:21.151810 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dshdc" event={"ID":"469094bf-3160-4036-a837-37e55eaa9323","Type":"ContainerStarted","Data":"32342022d29b670fff6f7603931d63d562faefc3016f11b1b6988d53046b5947"} Jul 14 19:38:21.171634 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:21.171575 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dshdc" podStartSLOduration=4.283719742 podStartE2EDuration="24.171558222s" totalImagesPullingTime="19.88783848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:21.170958238 +0000 UTC m=+24.645056978" watchObservedRunningTime="2026-07-14 19:38:21.171558222 +0000 UTC m=+24.645656961" Jul 14 19:38:22.040078 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.039838 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:22.040268 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:22.040195 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:22.050561 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.050521 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:38:22.051132 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.051111 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:38:22.156114 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.156079 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" event={"ID":"5572ed36-c92e-474c-b738-f962ab34ae1a","Type":"ContainerStarted","Data":"6b4bb73737d35e93d73d7410666ccb3a38195051ce6f2ac0075eefd8a93c695f"} Jul 14 19:38:22.159486 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.159462 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:38:22.159875 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.159841 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"97c570ecd59df89ad8466070eee88bcb3bf3e79d964ab0e68ca199a7b4c68f7e"} Jul 14 19:38:22.160270 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.160237 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:38:22.160750 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.160733 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-krgkr" Jul 14 19:38:22.182731 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:22.182676 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6xt7" podStartSLOduration=2.990582679 podStartE2EDuration="25.18265862s" totalImagesPullingTime="22.192075941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:22.182567047 +0000 UTC m=+25.656665787" watchObservedRunningTime="2026-07-14 19:38:22.18265862 +0000 UTC m=+25.656757360" Jul 14 19:38:23.039348 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:23.039313 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:23.039527 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:23.039459 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:23.039527 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:23.039512 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:23.039664 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:23.039620 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:24.040173 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:24.040132 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:24.040638 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:24.040277 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:25.040425 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.040170 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:25.040983 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.040231 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:25.040983 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:25.040551 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:25.040983 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:25.040587 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:25.168754 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.168674 2568 generic.go:356] "Generic (PLEG): container finished" podID="33d370ea-b86b-4e86-a173-0babca5f40e0" containerID="626dc3a2b568d6b78be404161c07f16113357665077b0607bd002d01fd14b6bf" exitCode=0 Jul 14 19:38:25.168903 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.168753 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerDied","Data":"626dc3a2b568d6b78be404161c07f16113357665077b0607bd002d01fd14b6bf"} Jul 14 19:38:25.172589 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.172570 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:38:25.172951 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.172922 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"c41913140a4f9340423e6eddeb43008a62c4141574578d1eac7f64000a602d51"} Jul 14 19:38:25.173248 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.173215 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:38:25.173323 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.173285 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:38:25.173414 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.173397 2568 scope.go:117] "RemoveContainer" containerID="673939624a210407011c67ddae2ef0d3a1428caae8a6b9573fc8cdb735c856b5" Jul 14 19:38:25.192244 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:25.192225 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:38:26.039374 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.039333 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:26.039663 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:26.039518 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:26.179166 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.179133 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:38:26.179598 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.179569 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" event={"ID":"d07927f8-3b5c-4f1d-a3e8-ad78915b6c10","Type":"ContainerStarted","Data":"dc458e5deba97cd44bf3f549fb28f9c47b301712f7051b9fcf7b31612f38b550"} Jul 14 19:38:26.179812 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.179784 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:38:26.196487 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.196456 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:38:26.216822 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.216766 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" podStartSLOduration=10.810857808 podStartE2EDuration="29.216746017s" totalImagesPullingTime="18.405888209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:26.216563236 +0000 UTC m=+29.690661978" watchObservedRunningTime="2026-07-14 19:38:26.216746017 +0000 UTC m=+29.690844760" Jul 14 19:38:26.714001 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.713717 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vg8jr"] Jul 14 19:38:26.714169 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.714084 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:26.714237 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:26.714216 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:26.715931 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.715907 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jq9xp"] Jul 14 19:38:26.716045 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.716008 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:26.716120 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:26.716101 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:26.716565 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.716533 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqk5f"] Jul 14 19:38:26.716661 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:26.716600 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:26.716707 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:26.716691 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:27.183804 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:27.183770 2568 generic.go:356] "Generic (PLEG): container finished" podID="33d370ea-b86b-4e86-a173-0babca5f40e0" containerID="0ef8e630b0852863249af6593f77d47c50ba1ec57f84e677ef498ea0249316ac" exitCode=0 Jul 14 19:38:27.184174 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:27.183858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerDied","Data":"0ef8e630b0852863249af6593f77d47c50ba1ec57f84e677ef498ea0249316ac"} Jul 14 19:38:28.040145 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:28.040103 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:28.040145 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:28.040153 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:28.040481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:28.040103 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:28.040481 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:28.040238 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:28.040481 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:28.040294 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:28.040481 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:28.040357 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:29.190879 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:29.190843 2568 generic.go:356] "Generic (PLEG): container finished" podID="33d370ea-b86b-4e86-a173-0babca5f40e0" containerID="41198d830601b65f1bff7800de5c74b398b1354a360187ddff5007fc09237087" exitCode=0 Jul 14 19:38:29.191701 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:29.190896 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerDied","Data":"41198d830601b65f1bff7800de5c74b398b1354a360187ddff5007fc09237087"} Jul 14 19:38:30.039183 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:30.039142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:30.039183 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:30.039187 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:30.042176 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.039627 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jq9xp" podUID="ec294756-8328-40b8-b4b1-36ca8c695ca5" Jul 14 19:38:30.042176 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:30.040077 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:30.042176 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.040217 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqk5f" podUID="df43b1ab-25e2-4928-80e8-91d3c6e80a59" Jul 14 19:38:30.042176 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.040336 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:38:30.826359 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:30.826118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:30.826856 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.826288 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:30.826856 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:30.826405 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:30.826856 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.826470 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret podName:df43b1ab-25e2-4928-80e8-91d3c6e80a59 nodeName:}" failed. No retries permitted until 2026-07-14 19:39:02.826447701 +0000 UTC m=+66.300546423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret") pod "global-pull-secret-syncer-rqk5f" (UID: "df43b1ab-25e2-4928-80e8-91d3c6e80a59") : object "kube-system"/"original-pull-secret" not registered Jul 14 19:38:30.826856 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.826526 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:30.826856 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.826584 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:39:02.826567575 +0000 UTC m=+66.300666317 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 19:38:30.927674 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:30.927634 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:30.927881 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.927854 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 19:38:30.927963 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.927887 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 19:38:30.927963 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.927901 2568 projected.go:196] Error preparing data for projected volume kube-api-access-vspqz for pod openshift-network-diagnostics/network-check-target-jq9xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:30.928035 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:30.927966 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz podName:ec294756-8328-40b8-b4b1-36ca8c695ca5 nodeName:}" failed. No retries permitted until 2026-07-14 19:39:02.927946688 +0000 UTC m=+66.402045409 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vspqz" (UniqueName: "kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz") pod "network-check-target-jq9xp" (UID: "ec294756-8328-40b8-b4b1-36ca8c695ca5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 19:38:31.804485 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.804452 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-126.ec2.internal" event="NodeReady" Jul 14 19:38:31.804639 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.804586 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 19:38:31.846174 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.846082 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54bbbc956-hhkq8"] Jul 14 19:38:31.851042 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.851016 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:31.855365 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.855322 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 19:38:31.855677 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.855655 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 19:38:31.855899 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.855876 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 19:38:31.855990 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.855971 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xdxqx\"" Jul 14 19:38:31.863446 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.863423 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 19:38:31.866265 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.866242 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54bbbc956-hhkq8"] Jul 14 19:38:31.869652 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.869608 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dzqfc"] Jul 14 19:38:31.873575 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.873554 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2zv28"] Jul 14 19:38:31.873703 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.873682 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dzqfc" Jul 14 19:38:31.877019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.876981 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cpr6w\"" Jul 14 19:38:31.877019 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.877011 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 19:38:31.877185 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.877167 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:31.877690 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.877672 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 19:38:31.877794 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.877750 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 19:38:31.881406 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.881136 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 19:38:31.881406 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.881151 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-774zx\"" Jul 14 19:38:31.881406 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.881248 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 19:38:31.890705 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.888856 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dzqfc"] Jul 14 19:38:31.890803 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:31.890715 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2zv28"] Jul 14 19:38:32.036788 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.036747 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-config-volume\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.036961 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.036800 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92vzj\" (UniqueName: \"kubernetes.io/projected/2137bf9d-7a42-4eb6-a328-3eb6418152ca-kube-api-access-92vzj\") pod \"ingress-canary-dzqfc\" (UID: \"2137bf9d-7a42-4eb6-a328-3eb6418152ca\") " pod="openshift-ingress-canary/ingress-canary-dzqfc" Jul 14 19:38:32.036961 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.036864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81f6f005-1d38-4805-b1cc-21c21d695785-ca-trust-extracted\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.036961 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.036888 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-trusted-ca\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.036961 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.036915 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-installation-pull-secrets\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.036961 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.036939 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-tmp-dir\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.036961 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.036961 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2137bf9d-7a42-4eb6-a328-3eb6418152ca-cert\") pod \"ingress-canary-dzqfc\" (UID: \"2137bf9d-7a42-4eb6-a328-3eb6418152ca\") " pod="openshift-ingress-canary/ingress-canary-dzqfc" Jul 14 19:38:32.037245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.037010 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.037245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.037034 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.037245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.037079 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-image-registry-private-configuration\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.037245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.037107 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sprd\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-kube-api-access-2sprd\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.037245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.037197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-registry-certificates\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.037245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.037232 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-bound-sa-token\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.037480 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.037273 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4htns\" (UniqueName: \"kubernetes.io/projected/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-kube-api-access-4htns\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.039921 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.039877 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:38:32.039921 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.039911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:38:32.040130 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.039896 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:38:32.043100 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.043060 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9cvvf\"" Jul 14 19:38:32.043258 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.043239 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 19:38:32.043504 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.043485 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 19:38:32.043743 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.043725 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 19:38:32.044042 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.044014 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cb2f5\"" Jul 14 19:38:32.044042 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.044021 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 19:38:32.138650 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-image-registry-private-configuration\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.138650 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sprd\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-kube-api-access-2sprd\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.138901 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-registry-certificates\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.138901 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138724 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-bound-sa-token\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.138901 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4htns\" (UniqueName: \"kubernetes.io/projected/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-kube-api-access-4htns\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.138901 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138782 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-config-volume\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.138901 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92vzj\" (UniqueName: \"kubernetes.io/projected/2137bf9d-7a42-4eb6-a328-3eb6418152ca-kube-api-access-92vzj\") pod \"ingress-canary-dzqfc\" (UID: \"2137bf9d-7a42-4eb6-a328-3eb6418152ca\") " pod="openshift-ingress-canary/ingress-canary-dzqfc" Jul 14 19:38:32.138901 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138861 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81f6f005-1d38-4805-b1cc-21c21d695785-ca-trust-extracted\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.138901 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-trusted-ca\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.139237 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-installation-pull-secrets\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.139237 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138941 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-tmp-dir\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.139237 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138963 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2137bf9d-7a42-4eb6-a328-3eb6418152ca-cert\") pod \"ingress-canary-dzqfc\" (UID: \"2137bf9d-7a42-4eb6-a328-3eb6418152ca\") " pod="openshift-ingress-canary/ingress-canary-dzqfc" Jul 14 19:38:32.139237 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.138994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.139237 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.139017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.139237 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.139188 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:38:32.139534 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.139260 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:38:32.639240866 +0000 UTC m=+36.113339606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:38:32.139534 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.139329 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81f6f005-1d38-4805-b1cc-21c21d695785-ca-trust-extracted\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.139534 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.139469 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-config-volume\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.139691 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.139559 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-registry-certificates\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.139691 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.139599 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:38:32.139691 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.139615 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:38:32.139691 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.139671 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:32.639654729 +0000 UTC m=+36.113753669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:38:32.139883 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.139807 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-tmp-dir\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.140056 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.140007 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-trusted-ca\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.144249 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.144222 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-installation-pull-secrets\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.144407 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.144263 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2137bf9d-7a42-4eb6-a328-3eb6418152ca-cert\") pod \"ingress-canary-dzqfc\" (UID: \"2137bf9d-7a42-4eb6-a328-3eb6418152ca\") " pod="openshift-ingress-canary/ingress-canary-dzqfc" Jul 14 19:38:32.145281 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.145251 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-image-registry-private-configuration\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.151912 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.151861 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92vzj\" (UniqueName: \"kubernetes.io/projected/2137bf9d-7a42-4eb6-a328-3eb6418152ca-kube-api-access-92vzj\") pod \"ingress-canary-dzqfc\" (UID: \"2137bf9d-7a42-4eb6-a328-3eb6418152ca\") " pod="openshift-ingress-canary/ingress-canary-dzqfc" Jul 14 19:38:32.151912 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.151882 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-bound-sa-token\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.152366 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.152338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4htns\" (UniqueName: \"kubernetes.io/projected/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-kube-api-access-4htns\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.153289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.153262 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sprd\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-kube-api-access-2sprd\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.185165 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.185126 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dzqfc" Jul 14 19:38:32.642938 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.642896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:32.642938 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:32.642935 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:32.643192 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.643051 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:38:32.643192 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.643060 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:38:32.643192 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.643084 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:38:32.643192 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.643128 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:38:33.643109996 +0000 UTC m=+37.117208717 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:38:32.643192 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:32.643147 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:33.64313825 +0000 UTC m=+37.117236969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:38:33.112022 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:33.111965 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dzqfc"] Jul 14 19:38:33.116243 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:38:33.116005 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2137bf9d_7a42_4eb6_a328_3eb6418152ca.slice/crio-e79d1b0c268bbeee70a7e0d0bdc6c5483600ee5a2c215acaacd6c1433217b2c1 WatchSource:0}: Error finding container e79d1b0c268bbeee70a7e0d0bdc6c5483600ee5a2c215acaacd6c1433217b2c1: Status 404 returned error can't find the container with id e79d1b0c268bbeee70a7e0d0bdc6c5483600ee5a2c215acaacd6c1433217b2c1 Jul 14 19:38:33.202681 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:33.202650 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dzqfc" event={"ID":"2137bf9d-7a42-4eb6-a328-3eb6418152ca","Type":"ContainerStarted","Data":"e79d1b0c268bbeee70a7e0d0bdc6c5483600ee5a2c215acaacd6c1433217b2c1"} Jul 14 19:38:33.205580 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:33.205534 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerStarted","Data":"fd1f9b9fb8eb7dc829feba5981bbf1aded91138cce397aad3d7fa0fadb834745"} Jul 14 19:38:33.651584 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:33.651542 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:33.651584 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:33.651584 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:33.651815 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:33.651688 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:38:33.651815 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:33.651699 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:38:33.651815 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:33.651707 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:38:33.651815 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:33.651753 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:38:35.651739252 +0000 UTC m=+39.125837983 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:38:33.651815 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:33.651786 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:35.651759798 +0000 UTC m=+39.125858515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:38:34.210921 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:34.210880 2568 generic.go:356] "Generic (PLEG): container finished" podID="33d370ea-b86b-4e86-a173-0babca5f40e0" containerID="fd1f9b9fb8eb7dc829feba5981bbf1aded91138cce397aad3d7fa0fadb834745" exitCode=0 Jul 14 19:38:34.211477 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:34.210946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerDied","Data":"fd1f9b9fb8eb7dc829feba5981bbf1aded91138cce397aad3d7fa0fadb834745"} Jul 14 19:38:35.217212 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:35.217169 2568 generic.go:356] "Generic (PLEG): container finished" podID="33d370ea-b86b-4e86-a173-0babca5f40e0" containerID="3c36da5ca3f79a85cb3e340753e81ae1bdba725b7328224a82cce04cb1e1b825" exitCode=0 Jul 14 19:38:35.217842 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:35.217271 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerDied","Data":"3c36da5ca3f79a85cb3e340753e81ae1bdba725b7328224a82cce04cb1e1b825"} Jul 14 19:38:35.664389 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:35.664346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:35.664576 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:35.664429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:35.664576 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:35.664517 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:38:35.664576 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:35.664543 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:38:35.664576 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:35.664561 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:38:35.664801 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:35.664613 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:39.664591315 +0000 UTC m=+43.138690051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:38:35.664801 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:35.664632 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:38:39.664622137 +0000 UTC m=+43.138720864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:38:36.220702 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:36.220666 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dzqfc" event={"ID":"2137bf9d-7a42-4eb6-a328-3eb6418152ca","Type":"ContainerStarted","Data":"a805e8dc1ec6a45d2c420a0b20a0863514bc5be0edb6672bbb95f65c2909c2d2"} Jul 14 19:38:36.225229 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:36.225203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58jks" event={"ID":"33d370ea-b86b-4e86-a173-0babca5f40e0","Type":"ContainerStarted","Data":"bac4dd233a50d7d423f8f89bf68229bc14e42a56aa1564b3bd619b40beb45174"} Jul 14 19:38:36.239824 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:36.239776 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dzqfc" podStartSLOduration=2.357410176 podStartE2EDuration="5.239760222s" totalImagesPullingTime="2.882350046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:36.239000337 +0000 UTC m=+39.713099079" watchObservedRunningTime="2026-07-14 19:38:36.239760222 +0000 UTC m=+39.713858961" Jul 14 19:38:36.267352 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:36.267246 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-58jks" podStartSLOduration=8.481215063 podStartE2EDuration="39.26722556s" totalImagesPullingTime="30.786010497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:36.265051843 +0000 UTC m=+39.739150582" watchObservedRunningTime="2026-07-14 19:38:36.26722556 +0000 UTC m=+39.741324300" Jul 14 19:38:39.692559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:39.692515 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:39.692559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:39.692564 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:39.693003 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:39.692670 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:38:39.693003 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:39.692693 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:38:39.693003 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:39.692709 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:38:39.693003 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:39.692757 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:47.692737692 +0000 UTC m=+51.166836413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:38:39.693003 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:39.692772 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:38:47.69276597 +0000 UTC m=+51.166864688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:38:47.751521 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:47.751475 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:38:47.751521 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:47.751525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:38:47.752070 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:47.751630 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:38:47.752070 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:47.751652 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:38:47.752070 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:47.751721 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:39:03.751705095 +0000 UTC m=+67.225803816 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:38:47.752070 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:47.751742 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:38:47.752070 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:38:47.751797 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:39:03.751784594 +0000 UTC m=+67.225883336 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:38:51.050328 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.050292 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj"] Jul 14 19:38:51.067355 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.067327 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:38:51.071390 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.071346 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 14 19:38:51.072607 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.072580 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 19:38:51.072748 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.072698 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 19:38:51.072815 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.072797 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 19:38:51.072896 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.072867 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-nqbzv\"" Jul 14 19:38:51.078286 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.078260 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj"] Jul 14 19:38:51.176692 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.176645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpfrt\" (UniqueName: \"kubernetes.io/projected/494d1f37-ade0-4f0e-a1cf-dc6a74553757-kube-api-access-qpfrt\") pod \"managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj\" (UID: \"494d1f37-ade0-4f0e-a1cf-dc6a74553757\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:38:51.176692 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.176692 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/494d1f37-ade0-4f0e-a1cf-dc6a74553757-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj\" (UID: \"494d1f37-ade0-4f0e-a1cf-dc6a74553757\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:38:51.277847 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.277808 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpfrt\" (UniqueName: \"kubernetes.io/projected/494d1f37-ade0-4f0e-a1cf-dc6a74553757-kube-api-access-qpfrt\") pod \"managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj\" (UID: \"494d1f37-ade0-4f0e-a1cf-dc6a74553757\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:38:51.277847 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.277849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/494d1f37-ade0-4f0e-a1cf-dc6a74553757-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj\" (UID: \"494d1f37-ade0-4f0e-a1cf-dc6a74553757\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:38:51.281585 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.281563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/494d1f37-ade0-4f0e-a1cf-dc6a74553757-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj\" (UID: \"494d1f37-ade0-4f0e-a1cf-dc6a74553757\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:38:51.286965 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.286940 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpfrt\" (UniqueName: \"kubernetes.io/projected/494d1f37-ade0-4f0e-a1cf-dc6a74553757-kube-api-access-qpfrt\") pod \"managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj\" (UID: \"494d1f37-ade0-4f0e-a1cf-dc6a74553757\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:38:51.387553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.387461 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:38:51.518684 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:51.518653 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj"] Jul 14 19:38:51.522349 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:38:51.522318 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod494d1f37_ade0_4f0e_a1cf_dc6a74553757.slice/crio-76dc0b66a999157dff66f6533495b516059d70e79697dc67b477010e80bc9437 WatchSource:0}: Error finding container 76dc0b66a999157dff66f6533495b516059d70e79697dc67b477010e80bc9437: Status 404 returned error can't find the container with id 76dc0b66a999157dff66f6533495b516059d70e79697dc67b477010e80bc9437 Jul 14 19:38:52.263400 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:52.263344 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" event={"ID":"494d1f37-ade0-4f0e-a1cf-dc6a74553757","Type":"ContainerStarted","Data":"76dc0b66a999157dff66f6533495b516059d70e79697dc67b477010e80bc9437"} Jul 14 19:38:54.269574 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:54.269530 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" event={"ID":"494d1f37-ade0-4f0e-a1cf-dc6a74553757","Type":"ContainerStarted","Data":"366b94b879140be97b965e52453c5eed3dd45d8092a3f177ade1aa994ad0beb0"} Jul 14 19:38:54.288639 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:54.288591 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" podStartSLOduration=0.749627217 podStartE2EDuration="3.288575947s" totalImagesPullingTime="2.53894873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:54.287936008 +0000 UTC m=+57.762034752" watchObservedRunningTime="2026-07-14 19:38:54.288575947 +0000 UTC m=+57.762674687" Jul 14 19:38:58.203005 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:38:58.202976 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7wbn" Jul 14 19:39:02.868870 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.868823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:39:02.869344 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.868881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:39:02.871810 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.871787 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 19:39:02.871869 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.871806 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 19:39:02.879277 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:02.879244 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 19:39:02.879363 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:02.879333 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:40:06.879312004 +0000 UTC m=+130.353410727 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : secret "metrics-daemon-secret" not found Jul 14 19:39:02.881527 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.881509 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df43b1ab-25e2-4928-80e8-91d3c6e80a59-original-pull-secret\") pod \"global-pull-secret-syncer-rqk5f\" (UID: \"df43b1ab-25e2-4928-80e8-91d3c6e80a59\") " pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:39:02.958667 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.958627 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqk5f" Jul 14 19:39:02.969308 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.969289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:39:02.972254 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.972226 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 19:39:02.982571 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.982529 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 19:39:02.992484 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:02.992456 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vspqz\" (UniqueName: \"kubernetes.io/projected/ec294756-8328-40b8-b4b1-36ca8c695ca5-kube-api-access-vspqz\") pod \"network-check-target-jq9xp\" (UID: \"ec294756-8328-40b8-b4b1-36ca8c695ca5\") " pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:39:03.088987 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:03.088959 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqk5f"] Jul 14 19:39:03.092434 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:39:03.092405 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf43b1ab_25e2_4928_80e8_91d3c6e80a59.slice/crio-d304aaa2123375bc7399ec044e44c8acb680a6e82d85d8be7a5ca0cc4f19caa3 WatchSource:0}: Error finding container d304aaa2123375bc7399ec044e44c8acb680a6e82d85d8be7a5ca0cc4f19caa3: Status 404 returned error can't find the container with id d304aaa2123375bc7399ec044e44c8acb680a6e82d85d8be7a5ca0cc4f19caa3 Jul 14 19:39:03.254337 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:03.254251 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cb2f5\"" Jul 14 19:39:03.261765 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:03.261736 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:39:03.292395 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:03.292353 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqk5f" event={"ID":"df43b1ab-25e2-4928-80e8-91d3c6e80a59","Type":"ContainerStarted","Data":"d304aaa2123375bc7399ec044e44c8acb680a6e82d85d8be7a5ca0cc4f19caa3"} Jul 14 19:39:03.385440 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:03.385411 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jq9xp"] Jul 14 19:39:03.388352 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:39:03.388325 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec294756_8328_40b8_b4b1_36ca8c695ca5.slice/crio-f7a2c186884bc88669cac73b8a382710ade08fac4cf555a69622f8340ace82a8 WatchSource:0}: Error finding container f7a2c186884bc88669cac73b8a382710ade08fac4cf555a69622f8340ace82a8: Status 404 returned error can't find the container with id f7a2c186884bc88669cac73b8a382710ade08fac4cf555a69622f8340ace82a8 Jul 14 19:39:03.777260 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:03.777220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:39:03.777487 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:03.777266 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:39:03.777487 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:03.777418 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:39:03.777487 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:03.777443 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:39:03.777644 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:03.777513 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:39:35.777490195 +0000 UTC m=+99.251588924 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:39:03.777644 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:03.777427 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:39:03.777644 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:03.777578 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:39:35.777562173 +0000 UTC m=+99.251660894 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:39:04.296510 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:04.296472 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jq9xp" event={"ID":"ec294756-8328-40b8-b4b1-36ca8c695ca5","Type":"ContainerStarted","Data":"f7a2c186884bc88669cac73b8a382710ade08fac4cf555a69622f8340ace82a8"} Jul 14 19:39:07.306554 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:07.306512 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqk5f" event={"ID":"df43b1ab-25e2-4928-80e8-91d3c6e80a59","Type":"ContainerStarted","Data":"234f22cfb587deee474d87883b795df87e0d7e1e0bf10fa29d67b7747af3ea45"} Jul 14 19:39:07.325481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:07.325408 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rqk5f" podStartSLOduration=66.606234415 podStartE2EDuration="1m10.325390092s" totalImagesPullingTime="3.719155677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:39:07.324556671 +0000 UTC m=+70.798655411" watchObservedRunningTime="2026-07-14 19:39:07.325390092 +0000 UTC m=+70.799488822" Jul 14 19:39:10.316283 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:10.316246 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jq9xp" event={"ID":"ec294756-8328-40b8-b4b1-36ca8c695ca5","Type":"ContainerStarted","Data":"45da2e7fe204ccaf43f45fcb8b2ea6bf0a4bf0c06e9a526559dba95fff9153a7"} Jul 14 19:39:10.316810 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:10.316370 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:39:10.334266 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:10.334214 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jq9xp" podStartSLOduration=67.362741901 podStartE2EDuration="1m13.334197252s" totalImagesPullingTime="5.971455351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:39:10.333336693 +0000 UTC m=+73.807435444" watchObservedRunningTime="2026-07-14 19:39:10.334197252 +0000 UTC m=+73.808295992" Jul 14 19:39:35.816760 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:35.816713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:39:35.817207 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:35.816764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:39:35.817207 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:35.816881 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:39:35.817207 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:35.816907 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:39:35.817207 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:35.816940 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:39:35.817207 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:35.816978 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:40:39.816962559 +0000 UTC m=+163.291061276 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:39:35.817207 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:39:35.816998 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:40:39.816984548 +0000 UTC m=+163.291083270 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:39:41.321108 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:39:41.321076 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jq9xp" Jul 14 19:40:06.949562 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:06.949506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:40:06.949968 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:06.949657 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 19:40:06.949968 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:06.949732 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs podName:81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922 nodeName:}" failed. No retries permitted until 2026-07-14 19:42:08.949716624 +0000 UTC m=+252.423815341 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs") pod "network-metrics-daemon-vg8jr" (UID: "81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922") : secret "metrics-daemon-secret" not found Jul 14 19:40:34.862359 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:34.862308 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" Jul 14 19:40:34.894191 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:34.894147 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2zv28" podUID="8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f" Jul 14 19:40:35.064189 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:35.064133 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vg8jr" podUID="81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922" Jul 14 19:40:35.519618 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:35.519587 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zv28" Jul 14 19:40:35.519796 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:35.519587 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:40:39.896508 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:39.896467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:40:39.896508 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:39.896506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:40:39.896946 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:39.896616 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 19:40:39.896946 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:39.896633 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 19:40:39.896946 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:39.896640 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbbc956-hhkq8: secret "image-registry-tls" not found Jul 14 19:40:39.896946 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:39.896706 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls podName:81f6f005-1d38-4805-b1cc-21c21d695785 nodeName:}" failed. No retries permitted until 2026-07-14 19:42:41.896689268 +0000 UTC m=+285.370787986 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls") pod "image-registry-54bbbc956-hhkq8" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785") : secret "image-registry-tls" not found Jul 14 19:40:39.896946 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:40:39.896719 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls podName:8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f nodeName:}" failed. No retries permitted until 2026-07-14 19:42:41.896713228 +0000 UTC m=+285.370811945 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls") pod "dns-default-2zv28" (UID: "8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f") : secret "dns-default-metrics-tls" not found Jul 14 19:40:49.039995 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:49.039948 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:40:54.565099 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:54.565063 2568 generic.go:356] "Generic (PLEG): container finished" podID="494d1f37-ade0-4f0e-a1cf-dc6a74553757" containerID="366b94b879140be97b965e52453c5eed3dd45d8092a3f177ade1aa994ad0beb0" exitCode=255 Jul 14 19:40:54.565574 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:54.565136 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" event={"ID":"494d1f37-ade0-4f0e-a1cf-dc6a74553757","Type":"ContainerDied","Data":"366b94b879140be97b965e52453c5eed3dd45d8092a3f177ade1aa994ad0beb0"} Jul 14 19:40:54.570897 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:54.570872 2568 scope.go:117] "RemoveContainer" containerID="366b94b879140be97b965e52453c5eed3dd45d8092a3f177ade1aa994ad0beb0" Jul 14 19:40:55.571927 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:40:55.571890 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" event={"ID":"494d1f37-ade0-4f0e-a1cf-dc6a74553757","Type":"ContainerStarted","Data":"e8c76589ff73e9886d57933fd9723ec6990d39270ede76eefdd60d7dd4f8a7d1"} Jul 14 19:41:37.333860 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.333811 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7krf7"] Jul 14 19:41:37.336798 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.336779 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-86f4bcd8c6-9lqpc"] Jul 14 19:41:37.336922 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.336901 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.339602 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.339582 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.340293 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.340272 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 19:41:37.340406 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.340295 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 19:41:37.342659 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.342639 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 19:41:37.342861 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.342848 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gw7zk\"" Jul 14 19:41:37.342918 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.342893 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5mkq6\"" Jul 14 19:41:37.342918 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.342910 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 14 19:41:37.343005 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.342932 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 14 19:41:37.343650 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.343635 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 19:41:37.344091 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.344074 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 14 19:41:37.344212 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.344140 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 19:41:37.346858 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.346836 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 14 19:41:37.348311 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.348292 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 19:41:37.349540 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.349524 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 19:41:37.361932 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.361896 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7krf7"] Jul 14 19:41:37.369760 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.369726 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86f4bcd8c6-9lqpc"] Jul 14 19:41:37.402832 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.402794 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8d0f259-c756-4852-af39-fd8dfd71282d-tmp\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.402832 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.402830 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.403055 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.402860 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d0f259-c756-4852-af39-fd8dfd71282d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.403055 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.402904 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.403055 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.402978 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-default-certificate\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.403055 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.403008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8d0f259-c756-4852-af39-fd8dfd71282d-snapshots\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.403179 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.403096 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp4sg\" (UniqueName: \"kubernetes.io/projected/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-kube-api-access-gp4sg\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.403179 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.403133 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8d0f259-c756-4852-af39-fd8dfd71282d-serving-cert\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.403179 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.403152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckkx7\" (UniqueName: \"kubernetes.io/projected/f8d0f259-c756-4852-af39-fd8dfd71282d-kube-api-access-ckkx7\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.403289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.403177 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-stats-auth\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.403289 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.403203 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d0f259-c756-4852-af39-fd8dfd71282d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.427910 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.427876 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl"] Jul 14 19:41:37.430778 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.430759 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.438451 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.438427 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 14 19:41:37.438571 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.438427 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 14 19:41:37.438571 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.438553 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 14 19:41:37.438571 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.438513 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pj99x\"" Jul 14 19:41:37.438677 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.438492 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 14 19:41:37.454070 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.454038 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl"] Jul 14 19:41:37.504353 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.504353 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d0f259-c756-4852-af39-fd8dfd71282d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.504638 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504368 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.504638 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504431 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-default-certificate\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.504638 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8d0f259-c756-4852-af39-fd8dfd71282d-snapshots\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.504638 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:37.504569 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 19:41:37.504638 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:37.504577 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:38.004554244 +0000 UTC m=+221.478652976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : configmap references non-existent config key: service-ca.crt Jul 14 19:41:37.504869 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504646 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp4sg\" (UniqueName: \"kubernetes.io/projected/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-kube-api-access-gp4sg\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.504869 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504683 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpfpd\" (UniqueName: \"kubernetes.io/projected/d638838d-ea06-4909-8d9c-2c8509cb5f18-kube-api-access-zpfpd\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.504869 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504716 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8d0f259-c756-4852-af39-fd8dfd71282d-serving-cert\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.504869 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckkx7\" (UniqueName: \"kubernetes.io/projected/f8d0f259-c756-4852-af39-fd8dfd71282d-kube-api-access-ckkx7\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.504869 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:37.504764 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:38.004741662 +0000 UTC m=+221.478840391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : secret "router-metrics-certs-default" not found Jul 14 19:41:37.504869 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-stats-auth\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.504869 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d0f259-c756-4852-af39-fd8dfd71282d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.505280 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.504971 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8d0f259-c756-4852-af39-fd8dfd71282d-tmp\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.505280 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.505008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d638838d-ea06-4909-8d9c-2c8509cb5f18-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.505280 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.505043 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d638838d-ea06-4909-8d9c-2c8509cb5f18-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.505280 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.505071 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d0f259-c756-4852-af39-fd8dfd71282d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.505280 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.505153 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8d0f259-c756-4852-af39-fd8dfd71282d-snapshots\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.505508 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.505309 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8d0f259-c756-4852-af39-fd8dfd71282d-tmp\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.505799 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.505777 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d0f259-c756-4852-af39-fd8dfd71282d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.507067 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.507037 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8d0f259-c756-4852-af39-fd8dfd71282d-serving-cert\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.507166 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.507116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-default-certificate\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.507313 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.507291 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-stats-auth\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.519596 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.519565 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp4sg\" (UniqueName: \"kubernetes.io/projected/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-kube-api-access-gp4sg\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:37.520261 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.520243 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckkx7\" (UniqueName: \"kubernetes.io/projected/f8d0f259-c756-4852-af39-fd8dfd71282d-kube-api-access-ckkx7\") pod \"insights-operator-68d9dd94cf-7krf7\" (UID: \"f8d0f259-c756-4852-af39-fd8dfd71282d\") " pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.605501 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.605400 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpfpd\" (UniqueName: \"kubernetes.io/projected/d638838d-ea06-4909-8d9c-2c8509cb5f18-kube-api-access-zpfpd\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.605501 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.605457 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d638838d-ea06-4909-8d9c-2c8509cb5f18-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.605501 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.605475 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d638838d-ea06-4909-8d9c-2c8509cb5f18-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.606080 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.606058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d638838d-ea06-4909-8d9c-2c8509cb5f18-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.607782 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.607756 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d638838d-ea06-4909-8d9c-2c8509cb5f18-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.614525 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.614495 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpfpd\" (UniqueName: \"kubernetes.io/projected/d638838d-ea06-4909-8d9c-2c8509cb5f18-kube-api-access-zpfpd\") pod \"kube-storage-version-migrator-operator-799cb7f958-dwjfl\" (UID: \"d638838d-ea06-4909-8d9c-2c8509cb5f18\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.647755 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.647721 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" Jul 14 19:41:37.739755 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.739722 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" Jul 14 19:41:37.781116 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.781042 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7krf7"] Jul 14 19:41:37.784316 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:41:37.784280 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d0f259_c756_4852_af39_fd8dfd71282d.slice/crio-377d8296bf0a05e9a49d25e4f0c87d380739018d25fb8d6bc2895a162eeada4d WatchSource:0}: Error finding container 377d8296bf0a05e9a49d25e4f0c87d380739018d25fb8d6bc2895a162eeada4d: Status 404 returned error can't find the container with id 377d8296bf0a05e9a49d25e4f0c87d380739018d25fb8d6bc2895a162eeada4d Jul 14 19:41:37.879066 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:37.878982 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl"] Jul 14 19:41:37.881922 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:41:37.881898 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd638838d_ea06_4909_8d9c_2c8509cb5f18.slice/crio-68ef8a89ebedc80e12f4393bb2818e2182150de2ecfdde059a8cc8af25a6a747 WatchSource:0}: Error finding container 68ef8a89ebedc80e12f4393bb2818e2182150de2ecfdde059a8cc8af25a6a747: Status 404 returned error can't find the container with id 68ef8a89ebedc80e12f4393bb2818e2182150de2ecfdde059a8cc8af25a6a747 Jul 14 19:41:38.009237 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:38.009195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:38.009237 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:38.009242 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:38.009488 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:38.009375 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:39.009356679 +0000 UTC m=+222.483455397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : configmap references non-existent config key: service-ca.crt Jul 14 19:41:38.009488 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:38.009407 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 19:41:38.009488 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:38.009452 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:39.009439953 +0000 UTC m=+222.483538670 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : secret "router-metrics-certs-default" not found Jul 14 19:41:38.692474 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:38.692434 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" event={"ID":"d638838d-ea06-4909-8d9c-2c8509cb5f18","Type":"ContainerStarted","Data":"68ef8a89ebedc80e12f4393bb2818e2182150de2ecfdde059a8cc8af25a6a747"} Jul 14 19:41:38.693493 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:38.693457 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" event={"ID":"f8d0f259-c756-4852-af39-fd8dfd71282d","Type":"ContainerStarted","Data":"377d8296bf0a05e9a49d25e4f0c87d380739018d25fb8d6bc2895a162eeada4d"} Jul 14 19:41:39.017919 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:39.017797 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:39.017919 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:39.017864 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:39.018122 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:39.017974 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:41.017955837 +0000 UTC m=+224.492054556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : configmap references non-existent config key: service-ca.crt Jul 14 19:41:39.018122 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:39.018043 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 19:41:39.018122 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:39.018104 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:41.018088226 +0000 UTC m=+224.492186945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : secret "router-metrics-certs-default" not found Jul 14 19:41:41.032211 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:41.032167 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:41.032211 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:41.032224 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:41.032750 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:41.032396 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:45.032357728 +0000 UTC m=+228.506456455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : configmap references non-existent config key: service-ca.crt Jul 14 19:41:41.032750 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:41.032448 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 19:41:41.032750 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:41.032520 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:45.032498392 +0000 UTC m=+228.506597112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : secret "router-metrics-certs-default" not found Jul 14 19:41:41.704403 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:41.704299 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" event={"ID":"f8d0f259-c756-4852-af39-fd8dfd71282d","Type":"ContainerStarted","Data":"c2d7273d9699c4e5aa2a5a4c9a33f873c3c64ad10ca1595dc2455aff33eb17ab"} Jul 14 19:41:41.737833 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:41.737765 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" podStartSLOduration=1.165120952 podStartE2EDuration="4.737740238s" totalImagesPullingTime="3.572619286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:41.736706485 +0000 UTC m=+225.210805224" watchObservedRunningTime="2026-07-14 19:41:41.737740238 +0000 UTC m=+225.211838980" Jul 14 19:41:42.528413 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.528355 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bhclh"] Jul 14 19:41:42.532022 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.531992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.589731 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.589699 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rt2ws\"" Jul 14 19:41:42.597601 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.597576 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 19:41:42.624793 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.624758 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 19:41:42.644842 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.644802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.645033 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.644885 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.645033 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.644919 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-568pj\" (UniqueName: \"kubernetes.io/projected/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-api-access-568pj\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.645033 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.644955 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-crio-socket\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.645189 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.645121 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-data-volume\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.657255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.657211 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bhclh"] Jul 14 19:41:42.709973 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.709932 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" event={"ID":"d638838d-ea06-4909-8d9c-2c8509cb5f18","Type":"ContainerStarted","Data":"91838c5054a9dfbbe41b664b639795ed460356676a1fc9bfc40a29100fd589be"} Jul 14 19:41:42.744029 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.743971 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" podStartSLOduration=1.686445111 podStartE2EDuration="5.743953958s" totalImagesPullingTime="4.057508847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:42.738884121 +0000 UTC m=+226.212982861" watchObservedRunningTime="2026-07-14 19:41:42.743953958 +0000 UTC m=+226.218052697" Jul 14 19:41:42.745622 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.745599 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.745692 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.745649 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.745745 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:42.745710 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 19:41:42.745799 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.745756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-568pj\" (UniqueName: \"kubernetes.io/projected/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-api-access-568pj\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.745799 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:42.745764 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:43.245748564 +0000 UTC m=+226.719847296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : configmap "kube-rbac-proxy" not found Jul 14 19:41:42.745799 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:42.745776 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 19:41:42.745972 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:42.745840 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:43.245821881 +0000 UTC m=+226.719920599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : secret "insights-runtime-extractor-tls" not found Jul 14 19:41:42.745972 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.745883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-crio-socket\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.746310 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.746008 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-data-volume\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.746310 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.746006 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-crio-socket\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.750475 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.746767 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-data-volume\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:42.764003 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:42.763974 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-568pj\" (UniqueName: \"kubernetes.io/projected/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-api-access-568pj\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:43.250363 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:43.250324 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:43.250585 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:43.250387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:43.250585 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:43.250467 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 19:41:43.250585 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:43.250487 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 19:41:43.250585 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:43.250519 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:44.250505531 +0000 UTC m=+227.724604248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : configmap "kube-rbac-proxy" not found Jul 14 19:41:43.250585 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:43.250561 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:44.250544366 +0000 UTC m=+227.724643084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : secret "insights-runtime-extractor-tls" not found Jul 14 19:41:44.258924 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.258887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:44.259334 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.258937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:44.259334 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:44.259044 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 19:41:44.259334 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:44.259120 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:46.259104983 +0000 UTC m=+229.733203702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : secret "insights-runtime-extractor-tls" not found Jul 14 19:41:44.259334 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:44.259045 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 19:41:44.259334 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:44.259174 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:46.259163144 +0000 UTC m=+229.733261862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : configmap "kube-rbac-proxy" not found Jul 14 19:41:44.306256 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.306229 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9pmc8_5bd55968-8301-4303-aa1a-f9853a39015d/dns-node-resolver/0.log" Jul 14 19:41:44.922127 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.922084 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-lt27z"] Jul 14 19:41:44.925442 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.925418 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:44.929782 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.929752 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 14 19:41:44.930247 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.930223 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 14 19:41:44.930481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.930464 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zwflc\"" Jul 14 19:41:44.930564 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.930467 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 14 19:41:44.930564 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.930505 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 14 19:41:44.940436 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.940410 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-lt27z"] Jul 14 19:41:44.963266 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.963211 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3a749fa2-c865-4054-8ccf-ac53fa9874c9-signing-cabundle\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:44.963459 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.963280 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vdqx\" (UniqueName: \"kubernetes.io/projected/3a749fa2-c865-4054-8ccf-ac53fa9874c9-kube-api-access-6vdqx\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:44.963459 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:44.963347 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3a749fa2-c865-4054-8ccf-ac53fa9874c9-signing-key\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:45.064780 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.064736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3a749fa2-c865-4054-8ccf-ac53fa9874c9-signing-cabundle\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:45.064780 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.064793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vdqx\" (UniqueName: \"kubernetes.io/projected/3a749fa2-c865-4054-8ccf-ac53fa9874c9-kube-api-access-6vdqx\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:45.065049 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.064819 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:45.065049 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.064849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:45.065049 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.064872 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3a749fa2-c865-4054-8ccf-ac53fa9874c9-signing-key\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:45.065049 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:45.065005 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 19:41:45.065049 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:45.065022 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:53.064992001 +0000 UTC m=+236.539090722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : configmap references non-existent config key: service-ca.crt Jul 14 19:41:45.065304 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:45.065066 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:53.065050191 +0000 UTC m=+236.539148924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : secret "router-metrics-certs-default" not found Jul 14 19:41:45.065558 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.065536 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3a749fa2-c865-4054-8ccf-ac53fa9874c9-signing-cabundle\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:45.067449 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.067431 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3a749fa2-c865-4054-8ccf-ac53fa9874c9-signing-key\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:45.083514 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.083490 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vdqx\" (UniqueName: \"kubernetes.io/projected/3a749fa2-c865-4054-8ccf-ac53fa9874c9-kube-api-access-6vdqx\") pod \"service-ca-79f68b9f4f-lt27z\" (UID: \"3a749fa2-c865-4054-8ccf-ac53fa9874c9\") " pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:45.235260 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.235147 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" Jul 14 19:41:45.371088 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.371048 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-lt27z"] Jul 14 19:41:45.374738 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:41:45.374709 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a749fa2_c865_4054_8ccf_ac53fa9874c9.slice/crio-56c589d76c084a5ce1baebae079744d59185b5d34f6fbe60aa513d082bf207af WatchSource:0}: Error finding container 56c589d76c084a5ce1baebae079744d59185b5d34f6fbe60aa513d082bf207af: Status 404 returned error can't find the container with id 56c589d76c084a5ce1baebae079744d59185b5d34f6fbe60aa513d082bf207af Jul 14 19:41:45.701499 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.701474 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xd4sq_c889a0ed-dd2e-4229-9fca-3417dfdd901a/node-ca/0.log" Jul 14 19:41:45.718855 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:45.718814 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" event={"ID":"3a749fa2-c865-4054-8ccf-ac53fa9874c9","Type":"ContainerStarted","Data":"56c589d76c084a5ce1baebae079744d59185b5d34f6fbe60aa513d082bf207af"} Jul 14 19:41:46.107535 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:46.107499 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dzqfc_2137bf9d-7a42-4eb6-a328-3eb6418152ca/serve-healthcheck-canary/0.log" Jul 14 19:41:46.275394 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:46.275352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:46.275587 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:46.275433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:46.275587 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:46.275522 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 19:41:46.275709 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:46.275595 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:50.275579349 +0000 UTC m=+233.749678066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : secret "insights-runtime-extractor-tls" not found Jul 14 19:41:46.275709 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:46.275523 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 19:41:46.275709 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:46.275682 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:50.275665157 +0000 UTC m=+233.749763886 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : configmap "kube-rbac-proxy" not found Jul 14 19:41:47.725969 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:47.725874 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" event={"ID":"3a749fa2-c865-4054-8ccf-ac53fa9874c9","Type":"ContainerStarted","Data":"8f9fb6023f88065d32bd42415d95a67cd8b03f166f110455f39f148be213da7e"} Jul 14 19:41:50.311316 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:50.311269 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:50.311788 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:50.311330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:50.311788 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:50.311436 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 19:41:50.311788 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:50.311471 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 19:41:50.311788 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:50.311493 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:58.311477135 +0000 UTC m=+241.785575853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : configmap "kube-rbac-proxy" not found Jul 14 19:41:50.311788 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:50.311545 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls podName:fa95c13a-6a35-48a6-8bc8-49b85d1ef299 nodeName:}" failed. No retries permitted until 2026-07-14 19:41:58.31152595 +0000 UTC m=+241.785624682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhclh" (UID: "fa95c13a-6a35-48a6-8bc8-49b85d1ef299") : secret "insights-runtime-extractor-tls" not found Jul 14 19:41:53.134747 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:53.134705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:53.134747 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:53.134749 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:53.135313 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:41:53.134902 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle podName:ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2 nodeName:}" failed. No retries permitted until 2026-07-14 19:42:09.134876861 +0000 UTC m=+252.608975584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle") pod "router-default-86f4bcd8c6-9lqpc" (UID: "ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2") : configmap references non-existent config key: service-ca.crt Jul 14 19:41:53.137244 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:53.137225 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-metrics-certs\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:41:58.383475 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.383432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:58.383878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.383494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:58.384018 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.384000 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:58.385817 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.385784 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa95c13a-6a35-48a6-8bc8-49b85d1ef299-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhclh\" (UID: \"fa95c13a-6a35-48a6-8bc8-49b85d1ef299\") " pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:58.446035 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.446006 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rt2ws\"" Jul 14 19:41:58.453830 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.453802 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bhclh" Jul 14 19:41:58.593672 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.593612 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-lt27z" podStartSLOduration=12.530467929 podStartE2EDuration="14.593593309s" totalImagesPullingTime="2.06312538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:47.75505999 +0000 UTC m=+231.229158729" watchObservedRunningTime="2026-07-14 19:41:58.593593309 +0000 UTC m=+242.067692052" Jul 14 19:41:58.594163 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.594087 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bhclh"] Jul 14 19:41:58.596725 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:41:58.596699 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa95c13a_6a35_48a6_8bc8_49b85d1ef299.slice/crio-406e38fc0e6028cae71fb31c8bc0ce39d88cb3e0d72925e0efc52f109c442be0 WatchSource:0}: Error finding container 406e38fc0e6028cae71fb31c8bc0ce39d88cb3e0d72925e0efc52f109c442be0: Status 404 returned error can't find the container with id 406e38fc0e6028cae71fb31c8bc0ce39d88cb3e0d72925e0efc52f109c442be0 Jul 14 19:41:58.760553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:41:58.760467 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bhclh" event={"ID":"fa95c13a-6a35-48a6-8bc8-49b85d1ef299","Type":"ContainerStarted","Data":"406e38fc0e6028cae71fb31c8bc0ce39d88cb3e0d72925e0efc52f109c442be0"} Jul 14 19:42:02.776553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:02.776458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bhclh" event={"ID":"fa95c13a-6a35-48a6-8bc8-49b85d1ef299","Type":"ContainerStarted","Data":"79df73fcd24b6f5b072d4a06227cdf554350ba8e0915a9f52b6903cdaf5c7aaf"} Jul 14 19:42:03.781899 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:03.781855 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bhclh" event={"ID":"fa95c13a-6a35-48a6-8bc8-49b85d1ef299","Type":"ContainerStarted","Data":"d36fcde877ed96a4f28dec9ea4b49781c920c15ba45871863988a2ed2afeca93"} Jul 14 19:42:05.790561 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:05.790518 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bhclh" event={"ID":"fa95c13a-6a35-48a6-8bc8-49b85d1ef299","Type":"ContainerStarted","Data":"0c5d2068822475569ab141fda95aaa6894777bc34436768ae41d3b4354670e2a"} Jul 14 19:42:05.811745 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:05.811683 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bhclh" podStartSLOduration=16.939661781 podStartE2EDuration="23.811666364s" totalImagesPullingTime="6.872004583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:41:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:05.810345289 +0000 UTC m=+249.284444029" watchObservedRunningTime="2026-07-14 19:42:05.811666364 +0000 UTC m=+249.285765103" Jul 14 19:42:08.971452 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:08.971394 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:42:08.973689 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:08.973668 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922-metrics-certs\") pod \"network-metrics-daemon-vg8jr\" (UID: \"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922\") " pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:42:09.143272 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.143233 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9cvvf\"" Jul 14 19:42:09.150802 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.150778 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vg8jr" Jul 14 19:42:09.173394 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.173347 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:42:09.173918 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.173899 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2-service-ca-bundle\") pod \"router-default-86f4bcd8c6-9lqpc\" (UID: \"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2\") " pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:42:09.279127 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.279093 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vg8jr"] Jul 14 19:42:09.282017 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:09.281986 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d2c11c_e4ce_4f5d_8f9b_2b896a5ed922.slice/crio-9a1d71aa28cb6abba40293c83ecd64cf3705ea05b3e14eecc5f0846c6e82a23f WatchSource:0}: Error finding container 9a1d71aa28cb6abba40293c83ecd64cf3705ea05b3e14eecc5f0846c6e82a23f: Status 404 returned error can't find the container with id 9a1d71aa28cb6abba40293c83ecd64cf3705ea05b3e14eecc5f0846c6e82a23f Jul 14 19:42:09.455935 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.455904 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gw7zk\"" Jul 14 19:42:09.464330 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.464299 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:42:09.606073 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.606043 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86f4bcd8c6-9lqpc"] Jul 14 19:42:09.608428 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:09.608400 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac28f7ae_17e2_4cde_8f9b_3a942d3ddbe2.slice/crio-c03c5b8155ab691e251ba203c2f81b1df017db2b83266cb2e9f918af3286c0bf WatchSource:0}: Error finding container c03c5b8155ab691e251ba203c2f81b1df017db2b83266cb2e9f918af3286c0bf: Status 404 returned error can't find the container with id c03c5b8155ab691e251ba203c2f81b1df017db2b83266cb2e9f918af3286c0bf Jul 14 19:42:09.804419 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.804366 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" event={"ID":"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2","Type":"ContainerStarted","Data":"b9a517198e3e5863bbe6d32b441373c81a13ab0cebb6f331546c8b2d5b7665b6"} Jul 14 19:42:09.804623 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.804453 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" event={"ID":"ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2","Type":"ContainerStarted","Data":"c03c5b8155ab691e251ba203c2f81b1df017db2b83266cb2e9f918af3286c0bf"} Jul 14 19:42:09.805618 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.805582 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vg8jr" event={"ID":"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922","Type":"ContainerStarted","Data":"9a1d71aa28cb6abba40293c83ecd64cf3705ea05b3e14eecc5f0846c6e82a23f"} Jul 14 19:42:09.830718 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:09.830659 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" podStartSLOduration=32.830643985 podStartE2EDuration="32.830643985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:09.830127321 +0000 UTC m=+253.304226061" watchObservedRunningTime="2026-07-14 19:42:09.830643985 +0000 UTC m=+253.304742725" Jul 14 19:42:10.464913 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:10.464870 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:42:10.467872 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:10.467841 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:42:10.809947 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:10.809908 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:42:10.811372 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:10.811347 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-86f4bcd8c6-9lqpc" Jul 14 19:42:11.814234 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.814187 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vg8jr" event={"ID":"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922","Type":"ContainerStarted","Data":"b803307877e4252cf5e942da4c058368b063ddb2f169fc5c44ccdcce9127bd3f"} Jul 14 19:42:11.814651 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.814247 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vg8jr" event={"ID":"81d2c11c-e4ce-4f5d-8f9b-2b896a5ed922","Type":"ContainerStarted","Data":"7e5b69a032112548aeba8d759e55e40794394a9c1ad4aa64685adde975f1bb8e"} Jul 14 19:42:11.857501 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.857467 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fc6745bf-xqkzh"] Jul 14 19:42:11.860513 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.860483 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:11.864757 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.864733 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 19:42:11.864877 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.864810 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 19:42:11.864877 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.864830 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 19:42:11.865017 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.865001 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 19:42:11.865151 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.865134 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 19:42:11.865215 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.865159 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 19:42:11.865633 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.865612 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 19:42:11.865942 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.865925 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5tp2h\"" Jul 14 19:42:11.868332 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.868275 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vg8jr" podStartSLOduration=253.19901661 podStartE2EDuration="4m14.868255678s" totalImagesPullingTime="1.669239068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:11.867763658 +0000 UTC m=+255.341862401" watchObservedRunningTime="2026-07-14 19:42:11.868255678 +0000 UTC m=+255.342354420" Jul 14 19:42:11.885360 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.885331 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fc6745bf-xqkzh"] Jul 14 19:42:11.997737 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.997701 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-serving-cert\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:11.997737 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.997742 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-oauth-config\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:11.997737 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.997760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-console-config\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:11.998022 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.997806 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-service-ca\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:11.998022 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.997830 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq767\" (UniqueName: \"kubernetes.io/projected/b810b189-c60d-4091-902c-b7e1db2c315c-kube-api-access-nq767\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:11.998022 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:11.997856 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-oauth-serving-cert\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.098763 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.098660 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-serving-cert\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.098763 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.098708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-oauth-config\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.098763 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.098757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-console-config\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.099037 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.098799 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-service-ca\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.099037 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.098823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq767\" (UniqueName: \"kubernetes.io/projected/b810b189-c60d-4091-902c-b7e1db2c315c-kube-api-access-nq767\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.099037 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.098852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-oauth-serving-cert\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.099585 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.099564 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-console-config\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.099715 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.099591 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-oauth-serving-cert\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.099715 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.099596 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-service-ca\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.101286 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.101258 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-oauth-config\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.101435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.101393 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-serving-cert\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.110498 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.110474 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq767\" (UniqueName: \"kubernetes.io/projected/b810b189-c60d-4091-902c-b7e1db2c315c-kube-api-access-nq767\") pod \"console-7fc6745bf-xqkzh\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.170822 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.170779 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:12.304524 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.304490 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fc6745bf-xqkzh"] Jul 14 19:42:12.307245 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:12.307207 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb810b189_c60d_4091_902c_b7e1db2c315c.slice/crio-d0130b299e20e07a8c6c6365d34fa2f5f8f424dca917b92329c68dd74b03d432 WatchSource:0}: Error finding container d0130b299e20e07a8c6c6365d34fa2f5f8f424dca917b92329c68dd74b03d432: Status 404 returned error can't find the container with id d0130b299e20e07a8c6c6365d34fa2f5f8f424dca917b92329c68dd74b03d432 Jul 14 19:42:12.633839 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.633808 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv"] Jul 14 19:42:12.638092 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.638073 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" Jul 14 19:42:12.640981 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.640958 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7gmwl\"" Jul 14 19:42:12.641155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.641135 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 14 19:42:12.650972 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.650949 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv"] Jul 14 19:42:12.806115 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.806078 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d2aeb254-67df-4d5b-b0b1-541bed0b2d41-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nn9rv\" (UID: \"d2aeb254-67df-4d5b-b0b1-541bed0b2d41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" Jul 14 19:42:12.823779 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.823736 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc6745bf-xqkzh" event={"ID":"b810b189-c60d-4091-902c-b7e1db2c315c","Type":"ContainerStarted","Data":"d0130b299e20e07a8c6c6365d34fa2f5f8f424dca917b92329c68dd74b03d432"} Jul 14 19:42:12.907568 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.907470 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d2aeb254-67df-4d5b-b0b1-541bed0b2d41-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nn9rv\" (UID: \"d2aeb254-67df-4d5b-b0b1-541bed0b2d41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" Jul 14 19:42:12.909845 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.909821 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d2aeb254-67df-4d5b-b0b1-541bed0b2d41-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nn9rv\" (UID: \"d2aeb254-67df-4d5b-b0b1-541bed0b2d41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" Jul 14 19:42:12.948040 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:12.948003 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" Jul 14 19:42:13.094788 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:13.094761 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv"] Jul 14 19:42:13.096880 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:13.096853 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2aeb254_67df_4d5b_b0b1_541bed0b2d41.slice/crio-a8e6a35d15ddf2dd8e763f167217658342606eef2d9c12e00d2be0c11c5ec4a9 WatchSource:0}: Error finding container a8e6a35d15ddf2dd8e763f167217658342606eef2d9c12e00d2be0c11c5ec4a9: Status 404 returned error can't find the container with id a8e6a35d15ddf2dd8e763f167217658342606eef2d9c12e00d2be0c11c5ec4a9 Jul 14 19:42:13.828594 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:13.828554 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" event={"ID":"d2aeb254-67df-4d5b-b0b1-541bed0b2d41","Type":"ContainerStarted","Data":"a8e6a35d15ddf2dd8e763f167217658342606eef2d9c12e00d2be0c11c5ec4a9"} Jul 14 19:42:14.835286 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:14.835238 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" event={"ID":"d2aeb254-67df-4d5b-b0b1-541bed0b2d41","Type":"ContainerStarted","Data":"e020891045b90bf7cc78b28ad05a8bed3aa84e8c78c47eb5c46b0b50dd27c708"} Jul 14 19:42:14.835770 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:14.835460 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" Jul 14 19:42:14.840944 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:14.840908 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" Jul 14 19:42:14.859263 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:14.859192 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nn9rv" podStartSLOduration=1.6352219350000001 podStartE2EDuration="2.859173896s" totalImagesPullingTime="1.223951961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:14.857513671 +0000 UTC m=+258.331612413" watchObservedRunningTime="2026-07-14 19:42:14.859173896 +0000 UTC m=+258.333272635" Jul 14 19:42:15.716601 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.716566 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lbhcq"] Jul 14 19:42:15.719502 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.719475 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.723695 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.723666 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 19:42:15.723825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.723695 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 14 19:42:15.723825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.723728 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 14 19:42:15.723825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.723739 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-945xv\"" Jul 14 19:42:15.723825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.723742 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 19:42:15.723825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.723796 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 19:42:15.729339 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.729317 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lbhcq"] Jul 14 19:42:15.833647 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.833611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43409727-caf9-4a21-bdf6-78605de73b61-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.833647 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.833662 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43409727-caf9-4a21-bdf6-78605de73b61-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.833862 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.833696 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf6m7\" (UniqueName: \"kubernetes.io/projected/43409727-caf9-4a21-bdf6-78605de73b61-kube-api-access-nf6m7\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.833862 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.833790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43409727-caf9-4a21-bdf6-78605de73b61-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.840891 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.840854 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc6745bf-xqkzh" event={"ID":"b810b189-c60d-4091-902c-b7e1db2c315c","Type":"ContainerStarted","Data":"471c77fbe66ff9945d7d8411d2ecd74b84694dbfa5484bc5659d858b90a16700"} Jul 14 19:42:15.862290 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.862235 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fc6745bf-xqkzh" podStartSLOduration=1.423004192 podStartE2EDuration="4.862214788s" totalImagesPullingTime="3.439210596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:15.86139744 +0000 UTC m=+259.335496177" watchObservedRunningTime="2026-07-14 19:42:15.862214788 +0000 UTC m=+259.336313527" Jul 14 19:42:15.935072 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.935027 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf6m7\" (UniqueName: \"kubernetes.io/projected/43409727-caf9-4a21-bdf6-78605de73b61-kube-api-access-nf6m7\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.935271 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.935127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43409727-caf9-4a21-bdf6-78605de73b61-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.935271 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.935211 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43409727-caf9-4a21-bdf6-78605de73b61-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.935503 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.935275 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43409727-caf9-4a21-bdf6-78605de73b61-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.935965 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.935940 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43409727-caf9-4a21-bdf6-78605de73b61-metrics-client-ca\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.937706 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.937684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43409727-caf9-4a21-bdf6-78605de73b61-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.937801 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.937693 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43409727-caf9-4a21-bdf6-78605de73b61-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:15.944731 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:15.944701 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf6m7\" (UniqueName: \"kubernetes.io/projected/43409727-caf9-4a21-bdf6-78605de73b61-kube-api-access-nf6m7\") pod \"prometheus-operator-64785b6675-lbhcq\" (UID: \"43409727-caf9-4a21-bdf6-78605de73b61\") " pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:16.028959 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:16.028862 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" Jul 14 19:42:16.158125 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:16.158101 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-lbhcq"] Jul 14 19:42:16.160522 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:16.160496 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43409727_caf9_4a21_bdf6_78605de73b61.slice/crio-56cb95e81dc31f3868f7205a4306f7f2b9787a19efd395ca6c7218ddbd0fa5c6 WatchSource:0}: Error finding container 56cb95e81dc31f3868f7205a4306f7f2b9787a19efd395ca6c7218ddbd0fa5c6: Status 404 returned error can't find the container with id 56cb95e81dc31f3868f7205a4306f7f2b9787a19efd395ca6c7218ddbd0fa5c6 Jul 14 19:42:16.846273 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:16.846231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" event={"ID":"43409727-caf9-4a21-bdf6-78605de73b61","Type":"ContainerStarted","Data":"56cb95e81dc31f3868f7205a4306f7f2b9787a19efd395ca6c7218ddbd0fa5c6"} Jul 14 19:42:17.852067 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:17.852021 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" event={"ID":"43409727-caf9-4a21-bdf6-78605de73b61","Type":"ContainerStarted","Data":"fcfed1251c6d34767bcab911a8c6464418922385bafc460426a97d6050a17096"} Jul 14 19:42:17.852490 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:17.852096 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" event={"ID":"43409727-caf9-4a21-bdf6-78605de73b61","Type":"ContainerStarted","Data":"188b5772c4f372393f3051b1ee67ba9d579c113275cc899965eede0d32c3e577"} Jul 14 19:42:17.872413 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:17.872339 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-lbhcq" podStartSLOduration=1.733916259 podStartE2EDuration="2.87232077s" totalImagesPullingTime="1.138404511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:17.871997383 +0000 UTC m=+261.346096123" watchObservedRunningTime="2026-07-14 19:42:17.87232077 +0000 UTC m=+261.346419509" Jul 14 19:42:20.185054 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.184980 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6q697"] Jul 14 19:42:20.188565 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.188545 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.192902 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.192880 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 19:42:20.193114 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.192882 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mtsrq\"" Jul 14 19:42:20.193195 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.193000 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 19:42:20.193273 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.193003 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 19:42:20.376240 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376176 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-tls\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.376240 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376244 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.376472 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376291 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7472\" (UniqueName: \"kubernetes.io/projected/c5504cf7-d644-4f5a-9d2f-895717adfe7b-kube-api-access-g7472\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.376472 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376343 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-sys\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.376472 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-textfile\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.376589 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376485 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5504cf7-d644-4f5a-9d2f-895717adfe7b-metrics-client-ca\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.376589 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376515 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.376589 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376546 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-wtmp\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.376688 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.376610 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-root\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477703 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477703 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7472\" (UniqueName: \"kubernetes.io/projected/c5504cf7-d644-4f5a-9d2f-895717adfe7b-kube-api-access-g7472\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-sys\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477731 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-textfile\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5504cf7-d644-4f5a-9d2f-895717adfe7b-metrics-client-ca\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477765 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477789 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-wtmp\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477814 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-sys\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-root\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.477934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-tls\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.478310 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477964 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-wtmp\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.478310 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.477952 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c5504cf7-d644-4f5a-9d2f-895717adfe7b-root\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.478310 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:42:20.478006 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 14 19:42:20.478310 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:42:20.478087 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-tls podName:c5504cf7-d644-4f5a-9d2f-895717adfe7b nodeName:}" failed. No retries permitted until 2026-07-14 19:42:20.978053429 +0000 UTC m=+264.452152155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-tls") pod "node-exporter-6q697" (UID: "c5504cf7-d644-4f5a-9d2f-895717adfe7b") : secret "node-exporter-tls" not found Jul 14 19:42:20.478497 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.478313 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.478497 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.478421 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5504cf7-d644-4f5a-9d2f-895717adfe7b-metrics-client-ca\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.478590 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.478575 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-textfile\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.480143 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.480125 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.503421 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.503370 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7472\" (UniqueName: \"kubernetes.io/projected/c5504cf7-d644-4f5a-9d2f-895717adfe7b-kube-api-access-g7472\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.540638 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.540595 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78ff6c656c-gscwx"] Jul 14 19:42:20.544025 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.543998 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.552644 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.552619 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 19:42:20.558142 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.558110 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78ff6c656c-gscwx"] Jul 14 19:42:20.679081 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.679035 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-oauth-config\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.679278 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.679088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-trusted-ca-bundle\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.679278 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.679163 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-serving-cert\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.679278 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.679231 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-oauth-serving-cert\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.679278 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.679262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-console-config\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.679523 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.679352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-service-ca\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.679523 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.679438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-585mm\" (UniqueName: \"kubernetes.io/projected/d743d9c0-c0ff-4890-a352-0a56a901c349-kube-api-access-585mm\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.780307 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.780206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-service-ca\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.780307 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.780263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-585mm\" (UniqueName: \"kubernetes.io/projected/d743d9c0-c0ff-4890-a352-0a56a901c349-kube-api-access-585mm\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.780553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.780313 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-oauth-config\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.780553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.780340 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-trusted-ca-bundle\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.780553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.780370 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-serving-cert\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.780553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.780444 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-oauth-serving-cert\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.780553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.780480 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-console-config\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.781158 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.781121 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-oauth-serving-cert\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.781158 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.781122 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-service-ca\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.781310 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.781201 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-trusted-ca-bundle\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.781310 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.781227 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-console-config\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.783522 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.783501 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-oauth-config\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.783522 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.783510 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-serving-cert\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.789455 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.789428 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-585mm\" (UniqueName: \"kubernetes.io/projected/d743d9c0-c0ff-4890-a352-0a56a901c349-kube-api-access-585mm\") pod \"console-78ff6c656c-gscwx\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.854089 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.854046 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:20.981967 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.981926 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-tls\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.984206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.984176 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c5504cf7-d644-4f5a-9d2f-895717adfe7b-node-exporter-tls\") pod \"node-exporter-6q697\" (UID: \"c5504cf7-d644-4f5a-9d2f-895717adfe7b\") " pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:20.997689 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:20.997662 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78ff6c656c-gscwx"] Jul 14 19:42:21.000342 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:21.000314 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd743d9c0_c0ff_4890_a352_0a56a901c349.slice/crio-63e6695fcc36d710d302836b99d880923edbe347aeeb7422c709e79f1f05fb55 WatchSource:0}: Error finding container 63e6695fcc36d710d302836b99d880923edbe347aeeb7422c709e79f1f05fb55: Status 404 returned error can't find the container with id 63e6695fcc36d710d302836b99d880923edbe347aeeb7422c709e79f1f05fb55 Jul 14 19:42:21.098559 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.098454 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6q697" Jul 14 19:42:21.120688 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:21.120648 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5504cf7_d644_4f5a_9d2f_895717adfe7b.slice/crio-4a2f56f92c6ecd1bc123ce07b63f7dae66f20c49dcb99b817044a38a0ee80608 WatchSource:0}: Error finding container 4a2f56f92c6ecd1bc123ce07b63f7dae66f20c49dcb99b817044a38a0ee80608: Status 404 returned error can't find the container with id 4a2f56f92c6ecd1bc123ce07b63f7dae66f20c49dcb99b817044a38a0ee80608 Jul 14 19:42:21.256075 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.256041 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 19:42:21.260095 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.260071 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.265633 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.265601 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 19:42:21.267202 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267176 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 19:42:21.267326 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267203 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 19:42:21.267326 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267237 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 19:42:21.267326 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267275 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 19:42:21.267326 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267177 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 19:42:21.267546 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267325 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 19:42:21.267546 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267176 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 19:42:21.267546 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267427 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 19:42:21.267647 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.267625 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r5j7r\"" Jul 14 19:42:21.293944 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.293896 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 19:42:21.385229 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385229 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385169 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn872\" (UniqueName: \"kubernetes.io/projected/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-kube-api-access-qn872\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385229 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385192 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385535 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385243 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-config-out\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385535 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385285 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385535 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-web-config\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385535 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385365 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385535 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385427 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385535 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385455 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385535 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385801 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385801 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385644 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.385801 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.385680 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.486618 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486584 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.486774 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.486774 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486649 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn872\" (UniqueName: \"kubernetes.io/projected/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-kube-api-access-qn872\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.486774 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.486774 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-config-out\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.486774 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.486774 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-web-config\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.487055 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486915 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.487055 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.486966 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.487055 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.487003 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.487201 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.487054 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.487201 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.487091 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.487201 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.487140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.487500 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.487476 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.488492 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.488470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.489614 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.488902 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.490097 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.490053 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-web-config\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.490191 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.490161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.490264 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.490223 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.490662 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.490638 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.491067 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.491046 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-config-out\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.491321 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.491299 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.491553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.491534 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.492567 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.492548 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.492802 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.492782 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.496207 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.496183 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn872\" (UniqueName: \"kubernetes.io/projected/2a6a80ab-fe86-4c75-ba5d-3179cbd81298-kube-api-access-qn872\") pod \"alertmanager-main-0\" (UID: \"2a6a80ab-fe86-4c75-ba5d-3179cbd81298\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.570034 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.569997 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 19:42:21.760659 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.760619 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 19:42:21.764405 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:21.764356 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a6a80ab_fe86_4c75_ba5d_3179cbd81298.slice/crio-87ee8312879bc285f628e2d2cae4674f3c2b8a1e2c9e68cc95534c587694e86b WatchSource:0}: Error finding container 87ee8312879bc285f628e2d2cae4674f3c2b8a1e2c9e68cc95534c587694e86b: Status 404 returned error can't find the container with id 87ee8312879bc285f628e2d2cae4674f3c2b8a1e2c9e68cc95534c587694e86b Jul 14 19:42:21.871562 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.871515 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerStarted","Data":"87ee8312879bc285f628e2d2cae4674f3c2b8a1e2c9e68cc95534c587694e86b"} Jul 14 19:42:21.872834 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.872803 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6q697" event={"ID":"c5504cf7-d644-4f5a-9d2f-895717adfe7b","Type":"ContainerStarted","Data":"4a2f56f92c6ecd1bc123ce07b63f7dae66f20c49dcb99b817044a38a0ee80608"} Jul 14 19:42:21.874602 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.874577 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78ff6c656c-gscwx" event={"ID":"d743d9c0-c0ff-4890-a352-0a56a901c349","Type":"ContainerStarted","Data":"f5fb9e094ed1458d599df2b2dc257cbdc6014d48197ba7f672ad9c1e48923286"} Jul 14 19:42:21.874739 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.874617 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78ff6c656c-gscwx" event={"ID":"d743d9c0-c0ff-4890-a352-0a56a901c349","Type":"ContainerStarted","Data":"63e6695fcc36d710d302836b99d880923edbe347aeeb7422c709e79f1f05fb55"} Jul 14 19:42:21.914864 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:21.914798 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78ff6c656c-gscwx" podStartSLOduration=1.914780259 podStartE2EDuration="1.914780259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:21.913510639 +0000 UTC m=+265.387609381" watchObservedRunningTime="2026-07-14 19:42:21.914780259 +0000 UTC m=+265.388878999" Jul 14 19:42:22.171734 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:22.171491 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:22.171734 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:22.171547 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:22.173265 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:22.173237 2568 patch_prober.go:28] interesting pod/console-7fc6745bf-xqkzh container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.15:8443/health\": dial tcp 10.133.0.15:8443: connect: connection refused" start-of-body= Jul 14 19:42:22.173404 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:22.173289 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7fc6745bf-xqkzh" podUID="b810b189-c60d-4091-902c-b7e1db2c315c" containerName="console" probeResult="failure" output="Get \"https://10.133.0.15:8443/health\": dial tcp 10.133.0.15:8443: connect: connection refused" Jul 14 19:42:22.881253 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:22.881219 2568 generic.go:356] "Generic (PLEG): container finished" podID="c5504cf7-d644-4f5a-9d2f-895717adfe7b" containerID="af172175fee3e6c18a93dfbb1237969cb0a87aaf0cad8d461df3880a9f3ef095" exitCode=0 Jul 14 19:42:22.881672 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:22.881315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6q697" event={"ID":"c5504cf7-d644-4f5a-9d2f-895717adfe7b","Type":"ContainerDied","Data":"af172175fee3e6c18a93dfbb1237969cb0a87aaf0cad8d461df3880a9f3ef095"} Jul 14 19:42:22.883080 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:22.883054 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerStarted","Data":"c6df20548b1bab9f5eff51b1bc3547ff1b24ed4d7a66d69299d3357655ce943c"} Jul 14 19:42:23.888288 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:23.888246 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6q697" event={"ID":"c5504cf7-d644-4f5a-9d2f-895717adfe7b","Type":"ContainerStarted","Data":"9e46bcd2211d3ab619803832493555c81fb3506bbcf17c5edda2ccdfce3bb0a7"} Jul 14 19:42:23.888745 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:23.888314 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6q697" event={"ID":"c5504cf7-d644-4f5a-9d2f-895717adfe7b","Type":"ContainerStarted","Data":"966d111ae94165156142c58e39ba90bd2cee10d73cb89a72a1abb3b8309d4922"} Jul 14 19:42:23.889858 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:23.889833 2568 generic.go:356] "Generic (PLEG): container finished" podID="2a6a80ab-fe86-4c75-ba5d-3179cbd81298" containerID="c6df20548b1bab9f5eff51b1bc3547ff1b24ed4d7a66d69299d3357655ce943c" exitCode=0 Jul 14 19:42:23.889962 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:23.889885 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerDied","Data":"c6df20548b1bab9f5eff51b1bc3547ff1b24ed4d7a66d69299d3357655ce943c"} Jul 14 19:42:23.909795 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:23.909741 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6q697" podStartSLOduration=2.92269139 podStartE2EDuration="3.90972417s" totalImagesPullingTime="923.062462ms" totalInitContainerRuntime="63.970318ms" isStatefulPod=true podCreationTimestamp="2026-07-14 19:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:23.908153524 +0000 UTC m=+267.382252277" watchObservedRunningTime="2026-07-14 19:42:23.90972417 +0000 UTC m=+267.383822909" Jul 14 19:42:26.472846 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:26.472427 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fc6745bf-xqkzh"] Jul 14 19:42:26.905987 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:26.905896 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerStarted","Data":"c1f916a7bdfcd2221798ce14039e3f6ac190fbf7f845eabb50e4595912152afd"} Jul 14 19:42:26.906171 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:26.905994 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerStarted","Data":"e53142a832e6eac86af08aed38abcb4922aaf5daac9643b20a6f196dcf27f5ad"} Jul 14 19:42:26.906171 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:26.906021 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerStarted","Data":"bb16cf34497cd64474d1d16b453840699a68d62b0b3dd239ac69cfc77ed6141f"} Jul 14 19:42:26.906171 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:26.906045 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerStarted","Data":"6fe14e25a988655a982579fac51714211a1825c389c5c1dc8b1964f2fee653b4"} Jul 14 19:42:26.906171 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:26.906067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerStarted","Data":"1b8a7984b00ba470360d3b7db6e911cb00aab9a85306d46f584e42ff4f46fdfb"} Jul 14 19:42:27.913369 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:27.913336 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a6a80ab-fe86-4c75-ba5d-3179cbd81298","Type":"ContainerStarted","Data":"99b782b6f14aceb179876d9d21d7a49ab8bb1db44f772087d4828990efba00bf"} Jul 14 19:42:27.948099 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:27.947925 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.378777348 podStartE2EDuration="6.94790438s" totalImagesPullingTime="4.553085724s" totalInitContainerRuntime="16.041308ms" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:27.945761201 +0000 UTC m=+271.419859942" watchObservedRunningTime="2026-07-14 19:42:27.94790438 +0000 UTC m=+271.422003121" Jul 14 19:42:30.854766 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:30.854717 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:30.855163 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:30.854785 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:42:30.856064 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:30.856041 2568 patch_prober.go:28] interesting pod/console-78ff6c656c-gscwx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.18:8443/health\": dial tcp 10.133.0.18:8443: connect: connection refused" start-of-body= Jul 14 19:42:30.856142 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:30.856084 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-78ff6c656c-gscwx" podUID="d743d9c0-c0ff-4890-a352-0a56a901c349" containerName="console" probeResult="failure" output="Get \"https://10.133.0.18:8443/health\": dial tcp 10.133.0.18:8443: connect: connection refused" Jul 14 19:42:38.521135 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:42:38.521074 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2zv28" podUID="8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f" Jul 14 19:42:38.521135 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:42:38.521096 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" Jul 14 19:42:38.954296 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:38.954264 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:42:38.954516 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:38.954277 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zv28" Jul 14 19:42:40.854894 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:40.854854 2568 patch_prober.go:28] interesting pod/console-78ff6c656c-gscwx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.18:8443/health\": dial tcp 10.133.0.18:8443: connect: connection refused" start-of-body= Jul 14 19:42:40.855427 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:40.854923 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-78ff6c656c-gscwx" podUID="d743d9c0-c0ff-4890-a352-0a56a901c349" containerName="console" probeResult="failure" output="Get \"https://10.133.0.18:8443/health\": dial tcp 10.133.0.18:8443: connect: connection refused" Jul 14 19:42:41.978452 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:41.978418 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:42:41.978452 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:41.978453 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:42:41.980825 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:41.980804 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f-metrics-tls\") pod \"dns-default-2zv28\" (UID: \"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f\") " pod="openshift-dns/dns-default-2zv28" Jul 14 19:42:41.980898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:41.980852 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"image-registry-54bbbc956-hhkq8\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:42:42.259083 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.258987 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-774zx\"" Jul 14 19:42:42.259083 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.258986 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xdxqx\"" Jul 14 19:42:42.265544 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.265518 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:42:42.265619 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.265604 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zv28" Jul 14 19:42:42.458865 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.458833 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54bbbc956-hhkq8"] Jul 14 19:42:42.462746 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:42.462716 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f6f005_1d38_4805_b1cc_21c21d695785.slice/crio-b1efa93755b55fbf8473107f250e8b62d3bdffc71b40b4043a457ff91a35252c WatchSource:0}: Error finding container b1efa93755b55fbf8473107f250e8b62d3bdffc71b40b4043a457ff91a35252c: Status 404 returned error can't find the container with id b1efa93755b55fbf8473107f250e8b62d3bdffc71b40b4043a457ff91a35252c Jul 14 19:42:42.469231 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.469202 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2zv28"] Jul 14 19:42:42.471685 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:42:42.471648 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa3cc3a_0d69_4e37_85c0_71358bc6fc3f.slice/crio-252f5b050a4d504baee72e86c86725e62c4af9251d00e56cfd09b52e916d9f97 WatchSource:0}: Error finding container 252f5b050a4d504baee72e86c86725e62c4af9251d00e56cfd09b52e916d9f97: Status 404 returned error can't find the container with id 252f5b050a4d504baee72e86c86725e62c4af9251d00e56cfd09b52e916d9f97 Jul 14 19:42:42.970675 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.970638 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" event={"ID":"81f6f005-1d38-4805-b1cc-21c21d695785","Type":"ContainerStarted","Data":"800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c"} Jul 14 19:42:42.970892 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.970693 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" event={"ID":"81f6f005-1d38-4805-b1cc-21c21d695785","Type":"ContainerStarted","Data":"b1efa93755b55fbf8473107f250e8b62d3bdffc71b40b4043a457ff91a35252c"} Jul 14 19:42:42.970892 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.970742 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:42:42.971922 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:42.971898 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zv28" event={"ID":"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f","Type":"ContainerStarted","Data":"252f5b050a4d504baee72e86c86725e62c4af9251d00e56cfd09b52e916d9f97"} Jul 14 19:42:44.981948 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:44.981912 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zv28" event={"ID":"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f","Type":"ContainerStarted","Data":"453b4616b36a659223b4a6457b8888ecb66e84e1d673585d55e2f87555d9b010"} Jul 14 19:42:44.982402 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:44.981969 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zv28" event={"ID":"8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f","Type":"ContainerStarted","Data":"e0a1c6972acdea8a7347bbdab198c9ca6307196e6a139374830b002af9130144"} Jul 14 19:42:44.982402 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:44.982007 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2zv28" Jul 14 19:42:45.004068 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:45.004019 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" podStartSLOduration=288.003999884 podStartE2EDuration="4m48.003999884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:37:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:43.015156969 +0000 UTC m=+286.489255708" watchObservedRunningTime="2026-07-14 19:42:45.003999884 +0000 UTC m=+288.478098624" Jul 14 19:42:45.004349 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:45.004330 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2zv28" podStartSLOduration=252.148759854 podStartE2EDuration="4m14.004322806s" totalImagesPullingTime="1.855562952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:45.002195188 +0000 UTC m=+288.476293941" watchObservedRunningTime="2026-07-14 19:42:45.004322806 +0000 UTC m=+288.478421545" Jul 14 19:42:50.855337 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:50.855290 2568 patch_prober.go:28] interesting pod/console-78ff6c656c-gscwx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.18:8443/health\": dial tcp 10.133.0.18:8443: connect: connection refused" start-of-body= Jul 14 19:42:50.855759 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:50.855365 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-78ff6c656c-gscwx" podUID="d743d9c0-c0ff-4890-a352-0a56a901c349" containerName="console" probeResult="failure" output="Get \"https://10.133.0.18:8443/health\": dial tcp 10.133.0.18:8443: connect: connection refused" Jul 14 19:42:51.499520 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:51.499457 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7fc6745bf-xqkzh" podUID="b810b189-c60d-4091-902c-b7e1db2c315c" containerName="console" containerID="cri-o://471c77fbe66ff9945d7d8411d2ecd74b84694dbfa5484bc5659d858b90a16700" gracePeriod=15 Jul 14 19:42:52.008575 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:52.008539 2568 generic.go:356] "Generic (PLEG): container finished" podID="f8d0f259-c756-4852-af39-fd8dfd71282d" containerID="c2d7273d9699c4e5aa2a5a4c9a33f873c3c64ad10ca1595dc2455aff33eb17ab" exitCode=0 Jul 14 19:42:52.009039 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:52.008614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" event={"ID":"f8d0f259-c756-4852-af39-fd8dfd71282d","Type":"ContainerDied","Data":"c2d7273d9699c4e5aa2a5a4c9a33f873c3c64ad10ca1595dc2455aff33eb17ab"} Jul 14 19:42:52.009109 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:52.009062 2568 scope.go:117] "RemoveContainer" containerID="c2d7273d9699c4e5aa2a5a4c9a33f873c3c64ad10ca1595dc2455aff33eb17ab" Jul 14 19:42:53.013596 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:53.013559 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7krf7" event={"ID":"f8d0f259-c756-4852-af39-fd8dfd71282d","Type":"ContainerStarted","Data":"3d1694db89c2474da0ad9b3909431f3d44f783143716d30b479e824fd89ba5de"} Jul 14 19:42:54.714897 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:54.714873 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/init-config-reloader/0.log" Jul 14 19:42:54.915307 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:54.915279 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/alertmanager/0.log" Jul 14 19:42:54.988231 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:54.988161 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2zv28" Jul 14 19:42:55.031433 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.031393 2568 generic.go:356] "Generic (PLEG): container finished" podID="494d1f37-ade0-4f0e-a1cf-dc6a74553757" containerID="e8c76589ff73e9886d57933fd9723ec6990d39270ede76eefdd60d7dd4f8a7d1" exitCode=255 Jul 14 19:42:55.031602 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.031411 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" event={"ID":"494d1f37-ade0-4f0e-a1cf-dc6a74553757","Type":"ContainerDied","Data":"e8c76589ff73e9886d57933fd9723ec6990d39270ede76eefdd60d7dd4f8a7d1"} Jul 14 19:42:55.031602 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.031530 2568 scope.go:117] "RemoveContainer" containerID="366b94b879140be97b965e52453c5eed3dd45d8092a3f177ade1aa994ad0beb0" Jul 14 19:42:55.038871 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.038846 2568 scope.go:117] "RemoveContainer" containerID="e8c76589ff73e9886d57933fd9723ec6990d39270ede76eefdd60d7dd4f8a7d1" Jul 14 19:42:55.039043 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:42:55.039021 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj_open-cluster-management-agent-addon(494d1f37-ade0-4f0e-a1cf-dc6a74553757)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" podUID="494d1f37-ade0-4f0e-a1cf-dc6a74553757" Jul 14 19:42:55.130216 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.130183 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/config-reloader/0.log" Jul 14 19:42:55.314706 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.314652 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/kube-rbac-proxy-web/0.log" Jul 14 19:42:55.514919 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.514890 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/kube-rbac-proxy/0.log" Jul 14 19:42:55.715975 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.715893 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/kube-rbac-proxy-metric/0.log" Jul 14 19:42:55.913785 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:55.913750 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/prom-label-proxy/0.log" Jul 14 19:42:56.038545 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.038511 2568 generic.go:356] "Generic (PLEG): container finished" podID="b810b189-c60d-4091-902c-b7e1db2c315c" containerID="471c77fbe66ff9945d7d8411d2ecd74b84694dbfa5484bc5659d858b90a16700" exitCode=0 Jul 14 19:42:56.038738 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.038581 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc6745bf-xqkzh" event={"ID":"b810b189-c60d-4091-902c-b7e1db2c315c","Type":"ContainerDied","Data":"471c77fbe66ff9945d7d8411d2ecd74b84694dbfa5484bc5659d858b90a16700"} Jul 14 19:42:56.161001 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.160970 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:56.308470 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.308429 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-oauth-config\") pod \"b810b189-c60d-4091-902c-b7e1db2c315c\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " Jul 14 19:42:56.308654 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.308476 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-oauth-serving-cert\") pod \"b810b189-c60d-4091-902c-b7e1db2c315c\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " Jul 14 19:42:56.308654 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.308518 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq767\" (UniqueName: \"kubernetes.io/projected/b810b189-c60d-4091-902c-b7e1db2c315c-kube-api-access-nq767\") pod \"b810b189-c60d-4091-902c-b7e1db2c315c\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " Jul 14 19:42:56.308654 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.308576 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-serving-cert\") pod \"b810b189-c60d-4091-902c-b7e1db2c315c\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " Jul 14 19:42:56.308654 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.308600 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-service-ca\") pod \"b810b189-c60d-4091-902c-b7e1db2c315c\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " Jul 14 19:42:56.308654 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.308650 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-console-config\") pod \"b810b189-c60d-4091-902c-b7e1db2c315c\" (UID: \"b810b189-c60d-4091-902c-b7e1db2c315c\") " Jul 14 19:42:56.309229 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.308972 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b810b189-c60d-4091-902c-b7e1db2c315c" (UID: "b810b189-c60d-4091-902c-b7e1db2c315c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:42:56.309229 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.309019 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-service-ca" (OuterVolumeSpecName: "service-ca") pod "b810b189-c60d-4091-902c-b7e1db2c315c" (UID: "b810b189-c60d-4091-902c-b7e1db2c315c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:42:56.309229 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.309110 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-console-config" (OuterVolumeSpecName: "console-config") pod "b810b189-c60d-4091-902c-b7e1db2c315c" (UID: "b810b189-c60d-4091-902c-b7e1db2c315c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:42:56.310909 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.310877 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b810b189-c60d-4091-902c-b7e1db2c315c" (UID: "b810b189-c60d-4091-902c-b7e1db2c315c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:42:56.311006 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.310933 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b810b189-c60d-4091-902c-b7e1db2c315c-kube-api-access-nq767" (OuterVolumeSpecName: "kube-api-access-nq767") pod "b810b189-c60d-4091-902c-b7e1db2c315c" (UID: "b810b189-c60d-4091-902c-b7e1db2c315c"). InnerVolumeSpecName "kube-api-access-nq767". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:42:56.311006 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.310931 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b810b189-c60d-4091-902c-b7e1db2c315c" (UID: "b810b189-c60d-4091-902c-b7e1db2c315c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:42:56.409885 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.409842 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-oauth-config\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:42:56.409885 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.409874 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-oauth-serving-cert\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:42:56.409885 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.409886 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq767\" (UniqueName: \"kubernetes.io/projected/b810b189-c60d-4091-902c-b7e1db2c315c-kube-api-access-nq767\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:42:56.409885 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.409895 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b810b189-c60d-4091-902c-b7e1db2c315c-console-serving-cert\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:42:56.410155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.409904 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-service-ca\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:42:56.410155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.409913 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b810b189-c60d-4091-902c-b7e1db2c315c-console-config\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:42:56.964745 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.964705 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:42:56.965352 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.965332 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:42:56.972302 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:56.972277 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 19:42:57.045080 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:57.045049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc6745bf-xqkzh" event={"ID":"b810b189-c60d-4091-902c-b7e1db2c315c","Type":"ContainerDied","Data":"d0130b299e20e07a8c6c6365d34fa2f5f8f424dca917b92329c68dd74b03d432"} Jul 14 19:42:57.045080 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:57.045086 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc6745bf-xqkzh" Jul 14 19:42:57.045321 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:57.045097 2568 scope.go:117] "RemoveContainer" containerID="471c77fbe66ff9945d7d8411d2ecd74b84694dbfa5484bc5659d858b90a16700" Jul 14 19:42:57.078635 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:57.078562 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fc6745bf-xqkzh"] Jul 14 19:42:57.080532 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:57.080507 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fc6745bf-xqkzh"] Jul 14 19:42:57.314718 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:57.314692 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6q697_c5504cf7-d644-4f5a-9d2f-895717adfe7b/init-textfile/0.log" Jul 14 19:42:57.516197 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:57.516158 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6q697_c5504cf7-d644-4f5a-9d2f-895717adfe7b/node-exporter/0.log" Jul 14 19:42:57.717246 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:57.717167 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6q697_c5504cf7-d644-4f5a-9d2f-895717adfe7b/kube-rbac-proxy/0.log" Jul 14 19:42:59.044309 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:42:59.044261 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b810b189-c60d-4091-902c-b7e1db2c315c" path="/var/lib/kubelet/pods/b810b189-c60d-4091-902c-b7e1db2c315c/volumes" Jul 14 19:43:00.859040 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:00.859006 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:43:00.862791 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:00.862766 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:43:01.115361 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:01.115273 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-lbhcq_43409727-caf9-4a21-bdf6-78605de73b61/prometheus-operator/0.log" Jul 14 19:43:01.314319 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:01.314288 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-lbhcq_43409727-caf9-4a21-bdf6-78605de73b61/kube-rbac-proxy/0.log" Jul 14 19:43:01.388133 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:01.388043 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" Jul 14 19:43:01.394425 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:01.394398 2568 scope.go:117] "RemoveContainer" containerID="e8c76589ff73e9886d57933fd9723ec6990d39270ede76eefdd60d7dd4f8a7d1" Jul 14 19:43:01.394653 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:43:01.394626 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj_open-cluster-management-agent-addon(494d1f37-ade0-4f0e-a1cf-dc6a74553757)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" podUID="494d1f37-ade0-4f0e-a1cf-dc6a74553757" Jul 14 19:43:01.514473 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:01.514439 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-nn9rv_d2aeb254-67df-4d5b-b0b1-541bed0b2d41/prometheus-operator-admission-webhook/0.log" Jul 14 19:43:02.269707 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:02.269674 2568 patch_prober.go:28] interesting pod/image-registry-54bbbc956-hhkq8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 19:43:02.270112 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:02.269739 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 19:43:03.517037 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:03.517003 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78ff6c656c-gscwx_d743d9c0-c0ff-4890-a352-0a56a901c349/console/0.log" Jul 14 19:43:03.980614 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:03.980584 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:43:04.314193 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:04.314154 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zv28_8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f/dns/0.log" Jul 14 19:43:04.514575 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:04.514545 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zv28_8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f/kube-rbac-proxy/0.log" Jul 14 19:43:05.518106 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:05.518079 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9pmc8_5bd55968-8301-4303-aa1a-f9853a39015d/dns-node-resolver/0.log" Jul 14 19:43:06.114694 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:06.114665 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54bbbc956-hhkq8_81f6f005-1d38-4805-b1cc-21c21d695785/registry/0.log" Jul 14 19:43:06.916343 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:06.916313 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xd4sq_c889a0ed-dd2e-4229-9fca-3417dfdd901a/node-ca/0.log" Jul 14 19:43:06.925532 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:06.925497 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78ff6c656c-gscwx"] Jul 14 19:43:07.114281 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:07.114244 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86f4bcd8c6-9lqpc_ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2/router/0.log" Jul 14 19:43:07.314169 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:07.314124 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dzqfc_2137bf9d-7a42-4eb6-a328-3eb6418152ca/serve-healthcheck-canary/0.log" Jul 14 19:43:08.106033 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:08.105995 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54bbbc956-hhkq8"] Jul 14 19:43:14.115439 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:14.115401 2568 generic.go:356] "Generic (PLEG): container finished" podID="d638838d-ea06-4909-8d9c-2c8509cb5f18" containerID="91838c5054a9dfbbe41b664b639795ed460356676a1fc9bfc40a29100fd589be" exitCode=0 Jul 14 19:43:14.115912 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:14.115478 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" event={"ID":"d638838d-ea06-4909-8d9c-2c8509cb5f18","Type":"ContainerDied","Data":"91838c5054a9dfbbe41b664b639795ed460356676a1fc9bfc40a29100fd589be"} Jul 14 19:43:14.115912 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:14.115865 2568 scope.go:117] "RemoveContainer" containerID="91838c5054a9dfbbe41b664b639795ed460356676a1fc9bfc40a29100fd589be" Jul 14 19:43:15.120440 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:15.120402 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dwjfl" event={"ID":"d638838d-ea06-4909-8d9c-2c8509cb5f18","Type":"ContainerStarted","Data":"7637e05100998c647fd72d6ab2a6d4255132611171efcc96a3edce5ae5aeba1a"} Jul 14 19:43:17.041631 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:17.041607 2568 scope.go:117] "RemoveContainer" containerID="e8c76589ff73e9886d57933fd9723ec6990d39270ede76eefdd60d7dd4f8a7d1" Jul 14 19:43:17.129687 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:17.129653 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dbdb469f6-zrmnj" event={"ID":"494d1f37-ade0-4f0e-a1cf-dc6a74553757","Type":"ContainerStarted","Data":"4c644ac23a6b77f1ff91e1f21f92a9a2f609427e477b8bd3022d89d53aa9c3fb"} Jul 14 19:43:31.948955 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:31.948893 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78ff6c656c-gscwx" podUID="d743d9c0-c0ff-4890-a352-0a56a901c349" containerName="console" containerID="cri-o://f5fb9e094ed1458d599df2b2dc257cbdc6014d48197ba7f672ad9c1e48923286" gracePeriod=15 Jul 14 19:43:32.185349 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.185315 2568 generic.go:356] "Generic (PLEG): container finished" podID="d743d9c0-c0ff-4890-a352-0a56a901c349" containerID="f5fb9e094ed1458d599df2b2dc257cbdc6014d48197ba7f672ad9c1e48923286" exitCode=0 Jul 14 19:43:32.185520 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.185363 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78ff6c656c-gscwx" event={"ID":"d743d9c0-c0ff-4890-a352-0a56a901c349","Type":"ContainerDied","Data":"f5fb9e094ed1458d599df2b2dc257cbdc6014d48197ba7f672ad9c1e48923286"} Jul 14 19:43:32.185520 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.185435 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78ff6c656c-gscwx" event={"ID":"d743d9c0-c0ff-4890-a352-0a56a901c349","Type":"ContainerDied","Data":"63e6695fcc36d710d302836b99d880923edbe347aeeb7422c709e79f1f05fb55"} Jul 14 19:43:32.185520 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.185451 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63e6695fcc36d710d302836b99d880923edbe347aeeb7422c709e79f1f05fb55" Jul 14 19:43:32.197260 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.197238 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:43:32.294095 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294063 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-serving-cert\") pod \"d743d9c0-c0ff-4890-a352-0a56a901c349\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " Jul 14 19:43:32.294296 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294133 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-service-ca\") pod \"d743d9c0-c0ff-4890-a352-0a56a901c349\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " Jul 14 19:43:32.294296 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294165 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-oauth-serving-cert\") pod \"d743d9c0-c0ff-4890-a352-0a56a901c349\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " Jul 14 19:43:32.294296 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294186 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-oauth-config\") pod \"d743d9c0-c0ff-4890-a352-0a56a901c349\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " Jul 14 19:43:32.294296 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294210 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-console-config\") pod \"d743d9c0-c0ff-4890-a352-0a56a901c349\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " Jul 14 19:43:32.294296 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294245 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-trusted-ca-bundle\") pod \"d743d9c0-c0ff-4890-a352-0a56a901c349\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " Jul 14 19:43:32.294595 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294304 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-585mm\" (UniqueName: \"kubernetes.io/projected/d743d9c0-c0ff-4890-a352-0a56a901c349-kube-api-access-585mm\") pod \"d743d9c0-c0ff-4890-a352-0a56a901c349\" (UID: \"d743d9c0-c0ff-4890-a352-0a56a901c349\") " Jul 14 19:43:32.294690 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294661 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d743d9c0-c0ff-4890-a352-0a56a901c349" (UID: "d743d9c0-c0ff-4890-a352-0a56a901c349"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:43:32.294736 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294681 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-service-ca" (OuterVolumeSpecName: "service-ca") pod "d743d9c0-c0ff-4890-a352-0a56a901c349" (UID: "d743d9c0-c0ff-4890-a352-0a56a901c349"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:43:32.294736 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294717 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-console-config" (OuterVolumeSpecName: "console-config") pod "d743d9c0-c0ff-4890-a352-0a56a901c349" (UID: "d743d9c0-c0ff-4890-a352-0a56a901c349"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:43:32.294994 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.294969 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d743d9c0-c0ff-4890-a352-0a56a901c349" (UID: "d743d9c0-c0ff-4890-a352-0a56a901c349"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:43:32.296478 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.296432 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d743d9c0-c0ff-4890-a352-0a56a901c349" (UID: "d743d9c0-c0ff-4890-a352-0a56a901c349"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:43:32.296581 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.296472 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d743d9c0-c0ff-4890-a352-0a56a901c349-kube-api-access-585mm" (OuterVolumeSpecName: "kube-api-access-585mm") pod "d743d9c0-c0ff-4890-a352-0a56a901c349" (UID: "d743d9c0-c0ff-4890-a352-0a56a901c349"). InnerVolumeSpecName "kube-api-access-585mm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:43:32.296646 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.296568 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d743d9c0-c0ff-4890-a352-0a56a901c349" (UID: "d743d9c0-c0ff-4890-a352-0a56a901c349"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:43:32.395501 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.395465 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-oauth-serving-cert\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:32.395501 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.395496 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-oauth-config\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:32.395501 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.395507 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-console-config\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:32.395750 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.395515 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-trusted-ca-bundle\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:32.395750 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.395523 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-585mm\" (UniqueName: \"kubernetes.io/projected/d743d9c0-c0ff-4890-a352-0a56a901c349-kube-api-access-585mm\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:32.395750 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.395532 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d743d9c0-c0ff-4890-a352-0a56a901c349-console-serving-cert\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:32.395750 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:32.395542 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d743d9c0-c0ff-4890-a352-0a56a901c349-service-ca\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:33.127010 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:33.126948 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" containerName="registry" containerID="cri-o://800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c" gracePeriod=30 Jul 14 19:43:33.189394 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:33.189348 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78ff6c656c-gscwx" Jul 14 19:43:33.209149 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:33.209115 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78ff6c656c-gscwx"] Jul 14 19:43:33.213924 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:33.213895 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78ff6c656c-gscwx"] Jul 14 19:43:33.977062 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:33.977023 2568 patch_prober.go:28] interesting pod/image-registry-54bbbc956-hhkq8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.133.0.6:5000/healthz\": dial tcp 10.133.0.6:5000: connect: connection refused" start-of-body= Jul 14 19:43:33.977240 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:33.977084 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" containerName="registry" probeResult="failure" output="Get \"https://10.133.0.6:5000/healthz\": dial tcp 10.133.0.6:5000: connect: connection refused" Jul 14 19:43:34.380161 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.380134 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:43:34.411839 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.411788 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81f6f005-1d38-4805-b1cc-21c21d695785-ca-trust-extracted\") pod \"81f6f005-1d38-4805-b1cc-21c21d695785\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " Jul 14 19:43:34.412026 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.411845 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") pod \"81f6f005-1d38-4805-b1cc-21c21d695785\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " Jul 14 19:43:34.412026 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.411885 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-installation-pull-secrets\") pod \"81f6f005-1d38-4805-b1cc-21c21d695785\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " Jul 14 19:43:34.412026 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.411909 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-trusted-ca\") pod \"81f6f005-1d38-4805-b1cc-21c21d695785\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " Jul 14 19:43:34.412026 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.411957 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sprd\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-kube-api-access-2sprd\") pod \"81f6f005-1d38-4805-b1cc-21c21d695785\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " Jul 14 19:43:34.412026 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.411994 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-registry-certificates\") pod \"81f6f005-1d38-4805-b1cc-21c21d695785\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " Jul 14 19:43:34.413583 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.412041 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-image-registry-private-configuration\") pod \"81f6f005-1d38-4805-b1cc-21c21d695785\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " Jul 14 19:43:34.413583 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.412090 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-bound-sa-token\") pod \"81f6f005-1d38-4805-b1cc-21c21d695785\" (UID: \"81f6f005-1d38-4805-b1cc-21c21d695785\") " Jul 14 19:43:34.413583 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.412459 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "81f6f005-1d38-4805-b1cc-21c21d695785" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:43:34.413583 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.412662 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "81f6f005-1d38-4805-b1cc-21c21d695785" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:43:34.415152 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.415108 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "81f6f005-1d38-4805-b1cc-21c21d695785" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:43:34.415573 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.415548 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "81f6f005-1d38-4805-b1cc-21c21d695785" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:43:34.415782 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.415754 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "81f6f005-1d38-4805-b1cc-21c21d695785" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:43:34.415940 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.415915 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "81f6f005-1d38-4805-b1cc-21c21d695785" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:43:34.416007 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.415967 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-kube-api-access-2sprd" (OuterVolumeSpecName: "kube-api-access-2sprd") pod "81f6f005-1d38-4805-b1cc-21c21d695785" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785"). InnerVolumeSpecName "kube-api-access-2sprd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:43:34.421878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.421850 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f6f005-1d38-4805-b1cc-21c21d695785-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "81f6f005-1d38-4805-b1cc-21c21d695785" (UID: "81f6f005-1d38-4805-b1cc-21c21d695785"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 19:43:34.513653 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.513557 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-bound-sa-token\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:34.513653 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.513590 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81f6f005-1d38-4805-b1cc-21c21d695785-ca-trust-extracted\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:34.513653 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.513601 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-registry-tls\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:34.513653 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.513610 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-installation-pull-secrets\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:34.513653 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.513619 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-trusted-ca\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:34.513653 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.513627 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sprd\" (UniqueName: \"kubernetes.io/projected/81f6f005-1d38-4805-b1cc-21c21d695785-kube-api-access-2sprd\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:34.513653 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.513635 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81f6f005-1d38-4805-b1cc-21c21d695785-registry-certificates\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:34.513653 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:34.513645 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81f6f005-1d38-4805-b1cc-21c21d695785-image-registry-private-configuration\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:43:35.045117 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.045080 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d743d9c0-c0ff-4890-a352-0a56a901c349" path="/var/lib/kubelet/pods/d743d9c0-c0ff-4890-a352-0a56a901c349/volumes" Jul 14 19:43:35.197960 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.197921 2568 generic.go:356] "Generic (PLEG): container finished" podID="81f6f005-1d38-4805-b1cc-21c21d695785" containerID="800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c" exitCode=0 Jul 14 19:43:35.197960 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.197967 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" Jul 14 19:43:35.198214 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.197960 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" event={"ID":"81f6f005-1d38-4805-b1cc-21c21d695785","Type":"ContainerDied","Data":"800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c"} Jul 14 19:43:35.198214 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.198077 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54bbbc956-hhkq8" event={"ID":"81f6f005-1d38-4805-b1cc-21c21d695785","Type":"ContainerDied","Data":"b1efa93755b55fbf8473107f250e8b62d3bdffc71b40b4043a457ff91a35252c"} Jul 14 19:43:35.198214 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.198098 2568 scope.go:117] "RemoveContainer" containerID="800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c" Jul 14 19:43:35.206658 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.206637 2568 scope.go:117] "RemoveContainer" containerID="800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c" Jul 14 19:43:35.207001 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:43:35.206979 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c\": container with ID starting with 800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c not found: ID does not exist" containerID="800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c" Jul 14 19:43:35.207066 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.207013 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c"} err="failed to get container status \"800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c\": rpc error: code = NotFound desc = could not find container \"800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c\": container with ID starting with 800cd08599b1a76d432db43fa84b6f97c9dbe40791a40dd0376c4e848c79c66c not found: ID does not exist" Jul 14 19:43:35.218101 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.218059 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54bbbc956-hhkq8"] Jul 14 19:43:35.224808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:35.224778 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-54bbbc956-hhkq8"] Jul 14 19:43:37.044097 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:37.044050 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" path="/var/lib/kubelet/pods/81f6f005-1d38-4805-b1cc-21c21d695785/volumes" Jul 14 19:43:44.601591 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601557 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5558b5987d-wbfrv"] Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601831 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" containerName="registry" Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601842 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" containerName="registry" Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601857 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b810b189-c60d-4091-902c-b7e1db2c315c" containerName="console" Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601862 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b810b189-c60d-4091-902c-b7e1db2c315c" containerName="console" Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601868 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d743d9c0-c0ff-4890-a352-0a56a901c349" containerName="console" Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601874 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d743d9c0-c0ff-4890-a352-0a56a901c349" containerName="console" Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601924 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="81f6f005-1d38-4805-b1cc-21c21d695785" containerName="registry" Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601930 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d743d9c0-c0ff-4890-a352-0a56a901c349" containerName="console" Jul 14 19:43:44.602065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.601936 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b810b189-c60d-4091-902c-b7e1db2c315c" containerName="console" Jul 14 19:43:44.607197 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.607170 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.610072 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.610042 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 19:43:44.610217 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.610150 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 19:43:44.610217 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.610201 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 19:43:44.610313 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.610293 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 19:43:44.610555 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.610533 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 19:43:44.610688 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.610592 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hkk2n\"" Jul 14 19:43:44.616221 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.616198 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 19:43:44.619766 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.619744 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5558b5987d-wbfrv"] Jul 14 19:43:44.693873 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.693832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.693873 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.693878 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-telemeter-client-tls\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.694086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.693901 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-federate-client-tls\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.694086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.693979 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-serving-certs-ca-bundle\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.694086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.694015 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.694086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.694047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-metrics-client-ca\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.694212 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.694088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw8nj\" (UniqueName: \"kubernetes.io/projected/7ed6a649-6f09-465e-889c-c8979b96f242-kube-api-access-zw8nj\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.694212 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.694124 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-secret-telemeter-client\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.794658 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.794619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw8nj\" (UniqueName: \"kubernetes.io/projected/7ed6a649-6f09-465e-889c-c8979b96f242-kube-api-access-zw8nj\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.794878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.794677 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-secret-telemeter-client\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.794878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.794714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.794878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.794744 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-telemeter-client-tls\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.794878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.794773 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-federate-client-tls\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.794878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.794801 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-serving-certs-ca-bundle\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.794878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.794826 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.794878 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.794860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-metrics-client-ca\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.795674 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.795649 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-metrics-client-ca\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.795787 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.795698 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-serving-certs-ca-bundle\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.795944 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.795918 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ed6a649-6f09-465e-889c-c8979b96f242-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.797525 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.797498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.797641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.797596 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-federate-client-tls\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.797769 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.797749 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-secret-telemeter-client\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.797847 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.797827 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7ed6a649-6f09-465e-889c-c8979b96f242-telemeter-client-tls\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.803406 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.803367 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw8nj\" (UniqueName: \"kubernetes.io/projected/7ed6a649-6f09-465e-889c-c8979b96f242-kube-api-access-zw8nj\") pod \"telemeter-client-5558b5987d-wbfrv\" (UID: \"7ed6a649-6f09-465e-889c-c8979b96f242\") " pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:44.918465 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:44.918363 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" Jul 14 19:43:45.062870 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:45.062832 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5558b5987d-wbfrv"] Jul 14 19:43:45.066074 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:43:45.066045 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed6a649_6f09_465e_889c_c8979b96f242.slice/crio-890b994ab31c05cd86dd6abbf1a3dbb9c6bf9c063e9c648dc393a5bb142cef10 WatchSource:0}: Error finding container 890b994ab31c05cd86dd6abbf1a3dbb9c6bf9c063e9c648dc393a5bb142cef10: Status 404 returned error can't find the container with id 890b994ab31c05cd86dd6abbf1a3dbb9c6bf9c063e9c648dc393a5bb142cef10 Jul 14 19:43:45.067953 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:45.067934 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 19:43:45.237734 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:45.237642 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" event={"ID":"7ed6a649-6f09-465e-889c-c8979b96f242","Type":"ContainerStarted","Data":"890b994ab31c05cd86dd6abbf1a3dbb9c6bf9c063e9c648dc393a5bb142cef10"} Jul 14 19:43:49.252579 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:49.252543 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" event={"ID":"7ed6a649-6f09-465e-889c-c8979b96f242","Type":"ContainerStarted","Data":"0ae4b28558d5ba3352225b47923da7bd74b1759ff31c5f846b34462dd0a742b7"} Jul 14 19:43:49.252906 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:49.252607 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" event={"ID":"7ed6a649-6f09-465e-889c-c8979b96f242","Type":"ContainerStarted","Data":"e14ab74daa6438c4e7cd069ef2c512a32fb3ae946bdda87e06d569e829f8c6b8"} Jul 14 19:43:50.258098 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:50.258054 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" event={"ID":"7ed6a649-6f09-465e-889c-c8979b96f242","Type":"ContainerStarted","Data":"c0d3c0ca4fae7b090f3779ccd48b7b6c7595b58a1974fdd91243a5c258275757"} Jul 14 19:43:50.288279 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:50.288223 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5558b5987d-wbfrv" podStartSLOduration=2.251132357 podStartE2EDuration="6.288207133s" totalImagesPullingTime="4.037074776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:43:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:43:50.286937465 +0000 UTC m=+353.761036196" watchObservedRunningTime="2026-07-14 19:43:50.288207133 +0000 UTC m=+353.762305870" Jul 14 19:43:51.089371 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.089339 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8b8654fb5-4sbhc"] Jul 14 19:43:51.092622 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.092598 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.096124 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.096097 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 19:43:51.096307 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.096294 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 19:43:51.096748 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.096728 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 19:43:51.097230 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.097212 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 19:43:51.097372 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.097231 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 19:43:51.097372 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.097328 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5tp2h\"" Jul 14 19:43:51.097570 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.097423 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 19:43:51.097623 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.097576 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 19:43:51.109062 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.109034 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 19:43:51.122533 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.122503 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b8654fb5-4sbhc"] Jul 14 19:43:51.251111 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.251074 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-serving-cert\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.251111 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.251124 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-config\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.251342 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.251198 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lsbv\" (UniqueName: \"kubernetes.io/projected/13f2478d-85e3-4879-b5d0-0c4a52f4a469-kube-api-access-9lsbv\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.251342 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.251268 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-oauth-serving-cert\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.251342 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.251296 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-service-ca\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.251342 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.251315 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-trusted-ca-bundle\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.251342 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.251335 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-oauth-config\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.352068 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.351969 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-config\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.352068 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.352034 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lsbv\" (UniqueName: \"kubernetes.io/projected/13f2478d-85e3-4879-b5d0-0c4a52f4a469-kube-api-access-9lsbv\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.352626 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.352085 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-oauth-serving-cert\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.352626 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.352116 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-service-ca\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.352626 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.352140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-trusted-ca-bundle\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.352626 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.352182 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-oauth-config\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.352626 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.352272 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-serving-cert\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.352922 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.352892 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-config\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.353217 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.353193 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-oauth-serving-cert\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.353306 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.353215 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-service-ca\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.353306 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.353236 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-trusted-ca-bundle\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.354957 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.354933 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-oauth-config\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.355092 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.355074 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-serving-cert\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.361134 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.361114 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lsbv\" (UniqueName: \"kubernetes.io/projected/13f2478d-85e3-4879-b5d0-0c4a52f4a469-kube-api-access-9lsbv\") pod \"console-8b8654fb5-4sbhc\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.402791 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.402745 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:43:51.541270 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:51.541210 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b8654fb5-4sbhc"] Jul 14 19:43:51.543334 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:43:51.543299 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f2478d_85e3_4879_b5d0_0c4a52f4a469.slice/crio-266f4017b8ebbd9ed2ab9c0e28cde885541bd17e9fe09f9726b6bb15be47ce0c WatchSource:0}: Error finding container 266f4017b8ebbd9ed2ab9c0e28cde885541bd17e9fe09f9726b6bb15be47ce0c: Status 404 returned error can't find the container with id 266f4017b8ebbd9ed2ab9c0e28cde885541bd17e9fe09f9726b6bb15be47ce0c Jul 14 19:43:52.273096 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:52.273060 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b8654fb5-4sbhc" event={"ID":"13f2478d-85e3-4879-b5d0-0c4a52f4a469","Type":"ContainerStarted","Data":"4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04"} Jul 14 19:43:52.273290 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:52.273135 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b8654fb5-4sbhc" event={"ID":"13f2478d-85e3-4879-b5d0-0c4a52f4a469","Type":"ContainerStarted","Data":"266f4017b8ebbd9ed2ab9c0e28cde885541bd17e9fe09f9726b6bb15be47ce0c"} Jul 14 19:43:52.295895 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:43:52.295846 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8b8654fb5-4sbhc" podStartSLOduration=1.295830055 podStartE2EDuration="1.295830055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:43:52.294439235 +0000 UTC m=+355.768537970" watchObservedRunningTime="2026-07-14 19:43:52.295830055 +0000 UTC m=+355.769928794" Jul 14 19:44:01.403322 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:01.403270 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:44:01.403730 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:01.403344 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:44:01.408790 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:01.408764 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:44:02.318453 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:02.318424 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:44:13.686442 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:13.686405 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b8654fb5-4sbhc"] Jul 14 19:44:38.706196 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:38.706155 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8b8654fb5-4sbhc" podUID="13f2478d-85e3-4879-b5d0-0c4a52f4a469" containerName="console" containerID="cri-o://4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04" gracePeriod=15 Jul 14 19:44:38.948961 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:38.948927 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:44:39.043994 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.043965 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-service-ca\") pod \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " Jul 14 19:44:39.044177 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044014 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-trusted-ca-bundle\") pod \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " Jul 14 19:44:39.044177 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044049 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-oauth-serving-cert\") pod \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " Jul 14 19:44:39.044177 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044075 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-oauth-config\") pod \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " Jul 14 19:44:39.044177 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044128 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lsbv\" (UniqueName: \"kubernetes.io/projected/13f2478d-85e3-4879-b5d0-0c4a52f4a469-kube-api-access-9lsbv\") pod \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " Jul 14 19:44:39.044177 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044160 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-config\") pod \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " Jul 14 19:44:39.044456 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044192 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-serving-cert\") pod \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\" (UID: \"13f2478d-85e3-4879-b5d0-0c4a52f4a469\") " Jul 14 19:44:39.044456 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044355 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-service-ca" (OuterVolumeSpecName: "service-ca") pod "13f2478d-85e3-4879-b5d0-0c4a52f4a469" (UID: "13f2478d-85e3-4879-b5d0-0c4a52f4a469"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:44:39.044562 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044513 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "13f2478d-85e3-4879-b5d0-0c4a52f4a469" (UID: "13f2478d-85e3-4879-b5d0-0c4a52f4a469"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:44:39.044660 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044624 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-config" (OuterVolumeSpecName: "console-config") pod "13f2478d-85e3-4879-b5d0-0c4a52f4a469" (UID: "13f2478d-85e3-4879-b5d0-0c4a52f4a469"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:44:39.044868 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.044840 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "13f2478d-85e3-4879-b5d0-0c4a52f4a469" (UID: "13f2478d-85e3-4879-b5d0-0c4a52f4a469"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 19:44:39.046414 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.046374 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "13f2478d-85e3-4879-b5d0-0c4a52f4a469" (UID: "13f2478d-85e3-4879-b5d0-0c4a52f4a469"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:44:39.046523 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.046435 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13f2478d-85e3-4879-b5d0-0c4a52f4a469-kube-api-access-9lsbv" (OuterVolumeSpecName: "kube-api-access-9lsbv") pod "13f2478d-85e3-4879-b5d0-0c4a52f4a469" (UID: "13f2478d-85e3-4879-b5d0-0c4a52f4a469"). InnerVolumeSpecName "kube-api-access-9lsbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:44:39.046582 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.046546 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "13f2478d-85e3-4879-b5d0-0c4a52f4a469" (UID: "13f2478d-85e3-4879-b5d0-0c4a52f4a469"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:44:39.145396 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.145352 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lsbv\" (UniqueName: \"kubernetes.io/projected/13f2478d-85e3-4879-b5d0-0c4a52f4a469-kube-api-access-9lsbv\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:44:39.145577 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.145409 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-config\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:44:39.145577 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.145423 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-serving-cert\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:44:39.145577 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.145434 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-service-ca\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:44:39.145577 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.145445 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-trusted-ca-bundle\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:44:39.145577 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.145456 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13f2478d-85e3-4879-b5d0-0c4a52f4a469-oauth-serving-cert\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:44:39.145577 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.145467 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13f2478d-85e3-4879-b5d0-0c4a52f4a469-console-oauth-config\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:44:39.457100 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.457011 2568 generic.go:356] "Generic (PLEG): container finished" podID="13f2478d-85e3-4879-b5d0-0c4a52f4a469" containerID="4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04" exitCode=0 Jul 14 19:44:39.457100 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.457058 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b8654fb5-4sbhc" Jul 14 19:44:39.457274 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.457095 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b8654fb5-4sbhc" event={"ID":"13f2478d-85e3-4879-b5d0-0c4a52f4a469","Type":"ContainerDied","Data":"4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04"} Jul 14 19:44:39.457274 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.457152 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b8654fb5-4sbhc" event={"ID":"13f2478d-85e3-4879-b5d0-0c4a52f4a469","Type":"ContainerDied","Data":"266f4017b8ebbd9ed2ab9c0e28cde885541bd17e9fe09f9726b6bb15be47ce0c"} Jul 14 19:44:39.457274 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.457177 2568 scope.go:117] "RemoveContainer" containerID="4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04" Jul 14 19:44:39.465931 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.465910 2568 scope.go:117] "RemoveContainer" containerID="4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04" Jul 14 19:44:39.466208 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:44:39.466189 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04\": container with ID starting with 4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04 not found: ID does not exist" containerID="4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04" Jul 14 19:44:39.466251 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.466216 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04"} err="failed to get container status \"4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04\": rpc error: code = NotFound desc = could not find container \"4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04\": container with ID starting with 4889aa779c50f76378ad0305238871d3cfee57a56d9b5be220201358aeaaed04 not found: ID does not exist" Jul 14 19:44:39.481927 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.481901 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b8654fb5-4sbhc"] Jul 14 19:44:39.490884 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:39.490861 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8b8654fb5-4sbhc"] Jul 14 19:44:41.044633 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:44:41.044595 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13f2478d-85e3-4879-b5d0-0c4a52f4a469" path="/var/lib/kubelet/pods/13f2478d-85e3-4879-b5d0-0c4a52f4a469/volumes" Jul 14 19:45:28.805658 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.805613 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm"] Jul 14 19:45:28.806257 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.806036 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f2478d-85e3-4879-b5d0-0c4a52f4a469" containerName="console" Jul 14 19:45:28.806257 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.806053 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f2478d-85e3-4879-b5d0-0c4a52f4a469" containerName="console" Jul 14 19:45:28.806257 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.806137 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="13f2478d-85e3-4879-b5d0-0c4a52f4a469" containerName="console" Jul 14 19:45:28.809048 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.809026 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" Jul 14 19:45:28.814534 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.814507 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 19:45:28.814669 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.814564 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sqzj8\"" Jul 14 19:45:28.815640 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.815619 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 19:45:28.823248 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.823217 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm"] Jul 14 19:45:28.838122 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.838087 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld5wv\" (UniqueName: \"kubernetes.io/projected/e9d7a7bd-5c03-4d8b-984e-931c981b39e2-kube-api-access-ld5wv\") pod \"cert-manager-cainjector-559cdd8c87-ddkdm\" (UID: \"e9d7a7bd-5c03-4d8b-984e-931c981b39e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" Jul 14 19:45:28.838279 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.838156 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9d7a7bd-5c03-4d8b-984e-931c981b39e2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ddkdm\" (UID: \"e9d7a7bd-5c03-4d8b-984e-931c981b39e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" Jul 14 19:45:28.938847 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.938807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld5wv\" (UniqueName: \"kubernetes.io/projected/e9d7a7bd-5c03-4d8b-984e-931c981b39e2-kube-api-access-ld5wv\") pod \"cert-manager-cainjector-559cdd8c87-ddkdm\" (UID: \"e9d7a7bd-5c03-4d8b-984e-931c981b39e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" Jul 14 19:45:28.938996 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.938896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9d7a7bd-5c03-4d8b-984e-931c981b39e2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ddkdm\" (UID: \"e9d7a7bd-5c03-4d8b-984e-931c981b39e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" Jul 14 19:45:28.947771 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.947737 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9d7a7bd-5c03-4d8b-984e-931c981b39e2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ddkdm\" (UID: \"e9d7a7bd-5c03-4d8b-984e-931c981b39e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" Jul 14 19:45:28.947898 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:28.947849 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld5wv\" (UniqueName: \"kubernetes.io/projected/e9d7a7bd-5c03-4d8b-984e-931c981b39e2-kube-api-access-ld5wv\") pod \"cert-manager-cainjector-559cdd8c87-ddkdm\" (UID: \"e9d7a7bd-5c03-4d8b-984e-931c981b39e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" Jul 14 19:45:29.122816 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:29.122723 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" Jul 14 19:45:29.258109 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:29.258076 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm"] Jul 14 19:45:29.261336 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:45:29.261299 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9d7a7bd_5c03_4d8b_984e_931c981b39e2.slice/crio-a6693e0474b2d6f2c49a21f511e648e7e895b2408af28a0b19a099e83c948ff3 WatchSource:0}: Error finding container a6693e0474b2d6f2c49a21f511e648e7e895b2408af28a0b19a099e83c948ff3: Status 404 returned error can't find the container with id a6693e0474b2d6f2c49a21f511e648e7e895b2408af28a0b19a099e83c948ff3 Jul 14 19:45:29.649110 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:29.649067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" event={"ID":"e9d7a7bd-5c03-4d8b-984e-931c981b39e2","Type":"ContainerStarted","Data":"a6693e0474b2d6f2c49a21f511e648e7e895b2408af28a0b19a099e83c948ff3"} Jul 14 19:45:32.665720 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:32.665679 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" event={"ID":"e9d7a7bd-5c03-4d8b-984e-931c981b39e2","Type":"ContainerStarted","Data":"cd1887e38fd8da7b7e8bebfe148d39f9d57e6bf5ef363393ee2618b33a8dd207"} Jul 14 19:45:32.697821 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:32.697768 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ddkdm" podStartSLOduration=1.755863546 podStartE2EDuration="4.697750133s" totalImagesPullingTime="2.941886587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:45:32.697202498 +0000 UTC m=+456.171301236" watchObservedRunningTime="2026-07-14 19:45:32.697750133 +0000 UTC m=+456.171848871" Jul 14 19:45:37.638199 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.638159 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2"] Jul 14 19:45:37.641455 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.641439 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" Jul 14 19:45:37.644268 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.644243 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 19:45:37.645014 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.644996 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-pqnqn\"" Jul 14 19:45:37.648287 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.648268 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 19:45:37.664868 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.664835 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2"] Jul 14 19:45:37.717347 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.717309 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4plb4\" (UniqueName: \"kubernetes.io/projected/07f18ef7-a29b-4175-a223-c95e881863cf-kube-api-access-4plb4\") pod \"openshift-lws-operator-fd8ccff4c-xmsb2\" (UID: \"07f18ef7-a29b-4175-a223-c95e881863cf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" Jul 14 19:45:37.717347 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.717353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07f18ef7-a29b-4175-a223-c95e881863cf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xmsb2\" (UID: \"07f18ef7-a29b-4175-a223-c95e881863cf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" Jul 14 19:45:37.818702 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.818662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4plb4\" (UniqueName: \"kubernetes.io/projected/07f18ef7-a29b-4175-a223-c95e881863cf-kube-api-access-4plb4\") pod \"openshift-lws-operator-fd8ccff4c-xmsb2\" (UID: \"07f18ef7-a29b-4175-a223-c95e881863cf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" Jul 14 19:45:37.818702 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.818707 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07f18ef7-a29b-4175-a223-c95e881863cf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xmsb2\" (UID: \"07f18ef7-a29b-4175-a223-c95e881863cf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" Jul 14 19:45:37.819132 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.819111 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07f18ef7-a29b-4175-a223-c95e881863cf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xmsb2\" (UID: \"07f18ef7-a29b-4175-a223-c95e881863cf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" Jul 14 19:45:37.866155 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.866120 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4plb4\" (UniqueName: \"kubernetes.io/projected/07f18ef7-a29b-4175-a223-c95e881863cf-kube-api-access-4plb4\") pod \"openshift-lws-operator-fd8ccff4c-xmsb2\" (UID: \"07f18ef7-a29b-4175-a223-c95e881863cf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" Jul 14 19:45:37.950091 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:37.949987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" Jul 14 19:45:38.098319 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:38.098293 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2"] Jul 14 19:45:38.101080 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:45:38.101051 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07f18ef7_a29b_4175_a223_c95e881863cf.slice/crio-a488204eae1ed8d7fc8d3df33a4b54cf55a6699c360bd6e918888dd48dfff87f WatchSource:0}: Error finding container a488204eae1ed8d7fc8d3df33a4b54cf55a6699c360bd6e918888dd48dfff87f: Status 404 returned error can't find the container with id a488204eae1ed8d7fc8d3df33a4b54cf55a6699c360bd6e918888dd48dfff87f Jul 14 19:45:38.688323 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:38.688289 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" event={"ID":"07f18ef7-a29b-4175-a223-c95e881863cf","Type":"ContainerStarted","Data":"a488204eae1ed8d7fc8d3df33a4b54cf55a6699c360bd6e918888dd48dfff87f"} Jul 14 19:45:41.703053 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:41.703012 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" event={"ID":"07f18ef7-a29b-4175-a223-c95e881863cf","Type":"ContainerStarted","Data":"d91a18eefb27d9b64823e72373d15f14d0946f150cd25c78062ceb3494ef288a"} Jul 14 19:45:41.740165 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:45:41.740110 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xmsb2" podStartSLOduration=2.006297385 podStartE2EDuration="4.740090533s" totalImagesPullingTime="2.733793148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:45:41.737539683 +0000 UTC m=+465.211638422" watchObservedRunningTime="2026-07-14 19:45:41.740090533 +0000 UTC m=+465.214189271" Jul 14 19:46:01.208920 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.208882 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc"] Jul 14 19:46:01.214957 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.214930 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.221699 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.221344 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 19:46:01.221847 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.221818 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 19:46:01.221967 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.221906 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 19:46:01.222583 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.222565 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 19:46:01.226507 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.226490 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wl8d8\"" Jul 14 19:46:01.240105 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.240070 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc"] Jul 14 19:46:01.311913 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.311880 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f112553b-066d-45af-892f-63185d600f0d-tmp\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.312100 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.311942 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f112553b-066d-45af-892f-63185d600f0d-webhook-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.312100 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.312006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f112553b-066d-45af-892f-63185d600f0d-manifests\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.312100 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.312043 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f112553b-066d-45af-892f-63185d600f0d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.312100 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.312066 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl7cr\" (UniqueName: \"kubernetes.io/projected/f112553b-066d-45af-892f-63185d600f0d-kube-api-access-zl7cr\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.412970 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.412933 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f112553b-066d-45af-892f-63185d600f0d-manifests\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.413191 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.412988 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f112553b-066d-45af-892f-63185d600f0d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.416236 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.413594 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl7cr\" (UniqueName: \"kubernetes.io/projected/f112553b-066d-45af-892f-63185d600f0d-kube-api-access-zl7cr\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.416236 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.413692 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f112553b-066d-45af-892f-63185d600f0d-tmp\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.416236 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.413734 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f112553b-066d-45af-892f-63185d600f0d-manifests\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.416236 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.413812 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f112553b-066d-45af-892f-63185d600f0d-webhook-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.419619 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.416712 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f112553b-066d-45af-892f-63185d600f0d-webhook-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.419619 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.416988 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f112553b-066d-45af-892f-63185d600f0d-tmp\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.419619 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.417128 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f112553b-066d-45af-892f-63185d600f0d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.423394 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.423361 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl7cr\" (UniqueName: \"kubernetes.io/projected/f112553b-066d-45af-892f-63185d600f0d-kube-api-access-zl7cr\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc\" (UID: \"f112553b-066d-45af-892f-63185d600f0d\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.525045 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.524955 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:01.679006 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.678981 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc"] Jul 14 19:46:01.681052 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:46:01.681023 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf112553b_066d_45af_892f_63185d600f0d.slice/crio-62716aa0c2a1cb83a12fec9b5ac6e3aa3377754426790fc20a732349389c1dd5 WatchSource:0}: Error finding container 62716aa0c2a1cb83a12fec9b5ac6e3aa3377754426790fc20a732349389c1dd5: Status 404 returned error can't find the container with id 62716aa0c2a1cb83a12fec9b5ac6e3aa3377754426790fc20a732349389c1dd5 Jul 14 19:46:01.784185 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:01.784101 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" event={"ID":"f112553b-066d-45af-892f-63185d600f0d","Type":"ContainerStarted","Data":"62716aa0c2a1cb83a12fec9b5ac6e3aa3377754426790fc20a732349389c1dd5"} Jul 14 19:46:05.806000 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:05.805962 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" event={"ID":"f112553b-066d-45af-892f-63185d600f0d","Type":"ContainerStarted","Data":"e22ba8ccddf387f296e8ba482be3e62c3ddc69ce08a5539d86ce15ed00ef640d"} Jul 14 19:46:06.812589 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:06.812549 2568 generic.go:356] "Generic (PLEG): container finished" podID="f112553b-066d-45af-892f-63185d600f0d" containerID="e22ba8ccddf387f296e8ba482be3e62c3ddc69ce08a5539d86ce15ed00ef640d" exitCode=0 Jul 14 19:46:06.812983 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:06.812636 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" event={"ID":"f112553b-066d-45af-892f-63185d600f0d","Type":"ContainerDied","Data":"e22ba8ccddf387f296e8ba482be3e62c3ddc69ce08a5539d86ce15ed00ef640d"} Jul 14 19:46:07.819328 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:07.819289 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" event={"ID":"f112553b-066d-45af-892f-63185d600f0d","Type":"ContainerStarted","Data":"d3021f310865ea4298a24e08de0590d72e03329809f489a261beec3dea70c0aa"} Jul 14 19:46:07.819821 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:07.819410 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:07.846489 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:07.846430 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" podStartSLOduration=1.973894777 podStartE2EDuration="6.846413994s" totalImagesPullingTime="3.859085635s" totalInitContainerRuntime="1.013433582s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:07.844971478 +0000 UTC m=+491.319070208" watchObservedRunningTime="2026-07-14 19:46:07.846413994 +0000 UTC m=+491.320512808" Jul 14 19:46:18.826361 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:18.826325 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc" Jul 14 19:46:22.457161 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.457117 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c"] Jul 14 19:46:22.460488 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.460471 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.467843 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.467824 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 19:46:22.468270 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.468254 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 19:46:22.484671 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.484637 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-tls-certs\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.484671 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.484670 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xglzw\" (UniqueName: \"kubernetes.io/projected/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-kube-api-access-xglzw\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.484847 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.484694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-tmp\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.491065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.491036 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c"] Jul 14 19:46:22.585678 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.585641 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-tls-certs\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.585678 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.585680 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglzw\" (UniqueName: \"kubernetes.io/projected/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-kube-api-access-xglzw\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.585952 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.585707 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-tmp\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.587950 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.587924 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-tmp\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.588168 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.588147 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-tls-certs\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.630814 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.630781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xglzw\" (UniqueName: \"kubernetes.io/projected/9efb5ff3-dca0-4a4b-9167-79b34afa5fcc-kube-api-access-xglzw\") pod \"kube-auth-proxy-59f575c9d9-rzq6c\" (UID: \"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.770042 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.769940 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" Jul 14 19:46:22.993328 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:46:22.993299 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efb5ff3_dca0_4a4b_9167_79b34afa5fcc.slice/crio-5b11c1dc5b24b3f696f95093d1d07723946b976ae68bfa2809e53081ce14e809 WatchSource:0}: Error finding container 5b11c1dc5b24b3f696f95093d1d07723946b976ae68bfa2809e53081ce14e809: Status 404 returned error can't find the container with id 5b11c1dc5b24b3f696f95093d1d07723946b976ae68bfa2809e53081ce14e809 Jul 14 19:46:22.993922 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:22.993900 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c"] Jul 14 19:46:23.883669 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:23.883633 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" event={"ID":"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc","Type":"ContainerStarted","Data":"5b11c1dc5b24b3f696f95093d1d07723946b976ae68bfa2809e53081ce14e809"} Jul 14 19:46:24.354665 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.354629 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6546g"] Jul 14 19:46:24.357795 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.357776 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:24.363135 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.363115 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 19:46:24.397797 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.397764 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6546g"] Jul 14 19:46:24.400609 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.400580 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45e5c6ea-d802-4624-a83b-611e583d4206-tls-certs\") pod \"model-serving-api-54d89bd5fc-6546g\" (UID: \"45e5c6ea-d802-4624-a83b-611e583d4206\") " pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:24.400743 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.400707 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6lnh\" (UniqueName: \"kubernetes.io/projected/45e5c6ea-d802-4624-a83b-611e583d4206-kube-api-access-z6lnh\") pod \"model-serving-api-54d89bd5fc-6546g\" (UID: \"45e5c6ea-d802-4624-a83b-611e583d4206\") " pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:24.501591 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.501552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45e5c6ea-d802-4624-a83b-611e583d4206-tls-certs\") pod \"model-serving-api-54d89bd5fc-6546g\" (UID: \"45e5c6ea-d802-4624-a83b-611e583d4206\") " pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:24.501797 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.501670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6lnh\" (UniqueName: \"kubernetes.io/projected/45e5c6ea-d802-4624-a83b-611e583d4206-kube-api-access-z6lnh\") pod \"model-serving-api-54d89bd5fc-6546g\" (UID: \"45e5c6ea-d802-4624-a83b-611e583d4206\") " pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:24.504449 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.504421 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45e5c6ea-d802-4624-a83b-611e583d4206-tls-certs\") pod \"model-serving-api-54d89bd5fc-6546g\" (UID: \"45e5c6ea-d802-4624-a83b-611e583d4206\") " pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:24.521030 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.520997 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6lnh\" (UniqueName: \"kubernetes.io/projected/45e5c6ea-d802-4624-a83b-611e583d4206-kube-api-access-z6lnh\") pod \"model-serving-api-54d89bd5fc-6546g\" (UID: \"45e5c6ea-d802-4624-a83b-611e583d4206\") " pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:24.668348 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.668254 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:24.877596 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.877548 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6546g"] Jul 14 19:46:24.881729 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:46:24.881696 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e5c6ea_d802_4624_a83b_611e583d4206.slice/crio-1ae31299d6065b2e521f3231c4f5763ae033dee25306bc77fc580b442955ea8c WatchSource:0}: Error finding container 1ae31299d6065b2e521f3231c4f5763ae033dee25306bc77fc580b442955ea8c: Status 404 returned error can't find the container with id 1ae31299d6065b2e521f3231c4f5763ae033dee25306bc77fc580b442955ea8c Jul 14 19:46:24.888876 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:24.888851 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6546g" event={"ID":"45e5c6ea-d802-4624-a83b-611e583d4206","Type":"ContainerStarted","Data":"1ae31299d6065b2e521f3231c4f5763ae033dee25306bc77fc580b442955ea8c"} Jul 14 19:46:26.901941 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:26.901899 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" event={"ID":"9efb5ff3-dca0-4a4b-9167-79b34afa5fcc","Type":"ContainerStarted","Data":"cae906250c41988a559ffbd0449f46889b303dc1ad92d2e92c3feafcd8274479"} Jul 14 19:46:26.952243 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:26.952179 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-rzq6c" podStartSLOduration=1.8962278320000001 podStartE2EDuration="4.952156573s" totalImagesPullingTime="3.055928741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:26.951122808 +0000 UTC m=+510.425221548" watchObservedRunningTime="2026-07-14 19:46:26.952156573 +0000 UTC m=+510.426255312" Jul 14 19:46:27.908715 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:27.908682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6546g" event={"ID":"45e5c6ea-d802-4624-a83b-611e583d4206","Type":"ContainerStarted","Data":"e28e15fbc4453e2d6dd8e14799156efaac9ec977d8faffeb5c400dc753a0f14d"} Jul 14 19:46:27.909113 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:27.908862 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:27.955884 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:27.955828 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6546g" podStartSLOduration=1.015752016 podStartE2EDuration="3.955810484s" totalImagesPullingTime="2.940058468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:27.955374198 +0000 UTC m=+511.429472940" watchObservedRunningTime="2026-07-14 19:46:27.955810484 +0000 UTC m=+511.429909222" Jul 14 19:46:30.693081 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.693045 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6"] Jul 14 19:46:30.696344 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.696326 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.701200 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.701174 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 19:46:30.701200 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.701181 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9kvsg\"" Jul 14 19:46:30.701435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.701218 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 19:46:30.701435 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.701181 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 19:46:30.709581 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.709560 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6"] Jul 14 19:46:30.753647 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.753613 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42dc27a2-e553-4c08-9f76-3a34d295cffa-cert\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.753647 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.753653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmzqh\" (UniqueName: \"kubernetes.io/projected/42dc27a2-e553-4c08-9f76-3a34d295cffa-kube-api-access-xmzqh\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.753942 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.753677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/42dc27a2-e553-4c08-9f76-3a34d295cffa-metrics-cert\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.753942 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.753787 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/42dc27a2-e553-4c08-9f76-3a34d295cffa-manager-config\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.855290 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.855237 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42dc27a2-e553-4c08-9f76-3a34d295cffa-cert\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.855290 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.855289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmzqh\" (UniqueName: \"kubernetes.io/projected/42dc27a2-e553-4c08-9f76-3a34d295cffa-kube-api-access-xmzqh\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.855546 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.855316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/42dc27a2-e553-4c08-9f76-3a34d295cffa-metrics-cert\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.855546 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.855350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/42dc27a2-e553-4c08-9f76-3a34d295cffa-manager-config\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.856143 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.856111 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/42dc27a2-e553-4c08-9f76-3a34d295cffa-manager-config\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.857875 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.857856 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42dc27a2-e553-4c08-9f76-3a34d295cffa-cert\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.857995 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.857974 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/42dc27a2-e553-4c08-9f76-3a34d295cffa-metrics-cert\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:30.873672 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:30.873639 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmzqh\" (UniqueName: \"kubernetes.io/projected/42dc27a2-e553-4c08-9f76-3a34d295cffa-kube-api-access-xmzqh\") pod \"lws-controller-manager-7f49b85c7f-cf8j6\" (UID: \"42dc27a2-e553-4c08-9f76-3a34d295cffa\") " pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:31.006197 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:31.006097 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:31.231925 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:31.231883 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6"] Jul 14 19:46:31.233885 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:46:31.233861 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42dc27a2_e553_4c08_9f76_3a34d295cffa.slice/crio-1ec37c982567464b08bbe7cf59be1e857066078a01dc1a39608bf8ed8f31a28c WatchSource:0}: Error finding container 1ec37c982567464b08bbe7cf59be1e857066078a01dc1a39608bf8ed8f31a28c: Status 404 returned error can't find the container with id 1ec37c982567464b08bbe7cf59be1e857066078a01dc1a39608bf8ed8f31a28c Jul 14 19:46:31.926844 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:31.926809 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" event={"ID":"42dc27a2-e553-4c08-9f76-3a34d295cffa","Type":"ContainerStarted","Data":"1ec37c982567464b08bbe7cf59be1e857066078a01dc1a39608bf8ed8f31a28c"} Jul 14 19:46:32.933184 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:32.933142 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" event={"ID":"42dc27a2-e553-4c08-9f76-3a34d295cffa","Type":"ContainerStarted","Data":"9411b28b051b30154feebd17969c0d5449e902c5bafe344b0b9a50f259236d5b"} Jul 14 19:46:32.933624 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:32.933302 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:46:32.978449 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:32.978394 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" podStartSLOduration=1.3999093409999999 podStartE2EDuration="2.978363322s" totalImagesPullingTime="1.578453981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:32.975865357 +0000 UTC m=+516.449964096" watchObservedRunningTime="2026-07-14 19:46:32.978363322 +0000 UTC m=+516.452462061" Jul 14 19:46:38.918137 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:38.918103 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6546g" Jul 14 19:46:43.940120 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:46:43.940089 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f49b85c7f-cf8j6" Jul 14 19:47:56.990403 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:47:56.990354 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:47:56.992552 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:47:56.992525 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:48:13.074015 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.073969 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c"] Jul 14 19:48:13.077194 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.077174 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.080652 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.080625 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 14 19:48:13.081202 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.081182 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 14 19:48:13.081952 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.081933 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 19:48:13.082023 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.081953 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 19:48:13.082023 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.081953 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j6wm7\"" Jul 14 19:48:13.110831 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.110795 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c"] Jul 14 19:48:13.146509 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.146464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/25cb1ca7-4b33-46b5-b139-afaed6751dd3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.146509 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.146514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/25cb1ca7-4b33-46b5-b139-afaed6751dd3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.146726 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.146546 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-489x8\" (UniqueName: \"kubernetes.io/projected/25cb1ca7-4b33-46b5-b139-afaed6751dd3-kube-api-access-489x8\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.247918 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.247871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/25cb1ca7-4b33-46b5-b139-afaed6751dd3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.247918 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.247922 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/25cb1ca7-4b33-46b5-b139-afaed6751dd3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.248157 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.247957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-489x8\" (UniqueName: \"kubernetes.io/projected/25cb1ca7-4b33-46b5-b139-afaed6751dd3-kube-api-access-489x8\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.248733 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.248711 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/25cb1ca7-4b33-46b5-b139-afaed6751dd3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.250675 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.250648 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/25cb1ca7-4b33-46b5-b139-afaed6751dd3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.262939 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.262906 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-489x8\" (UniqueName: \"kubernetes.io/projected/25cb1ca7-4b33-46b5-b139-afaed6751dd3-kube-api-access-489x8\") pod \"kuadrant-console-plugin-66b5774985-d2h6c\" (UID: \"25cb1ca7-4b33-46b5-b139-afaed6751dd3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.386756 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.386666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" Jul 14 19:48:13.525127 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:13.525098 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c"] Jul 14 19:48:13.527086 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:48:13.527058 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25cb1ca7_4b33_46b5_b139_afaed6751dd3.slice/crio-2a248ef60dc0afb53d088a9c9f40c1f2acb71dedb1c88c9605375ead3b3267d2 WatchSource:0}: Error finding container 2a248ef60dc0afb53d088a9c9f40c1f2acb71dedb1c88c9605375ead3b3267d2: Status 404 returned error can't find the container with id 2a248ef60dc0afb53d088a9c9f40c1f2acb71dedb1c88c9605375ead3b3267d2 Jul 14 19:48:14.383223 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:14.383187 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" event={"ID":"25cb1ca7-4b33-46b5-b139-afaed6751dd3","Type":"ContainerStarted","Data":"2a248ef60dc0afb53d088a9c9f40c1f2acb71dedb1c88c9605375ead3b3267d2"} Jul 14 19:48:38.510352 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:38.510260 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" event={"ID":"25cb1ca7-4b33-46b5-b139-afaed6751dd3","Type":"ContainerStarted","Data":"08ebfbf7d57eefe6694afcb3bf1228ef235fb1b861334a9c856366a5d3eb5925"} Jul 14 19:48:38.527251 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:38.527179 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2h6c" podStartSLOduration=0.850994449 podStartE2EDuration="25.52716246s" totalImagesPullingTime="24.676168011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:48:38.526616221 +0000 UTC m=+642.000714962" watchObservedRunningTime="2026-07-14 19:48:38.52716246 +0000 UTC m=+642.001261271" Jul 14 19:48:57.001026 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:48:57.000991 2568 scope.go:117] "RemoveContainer" containerID="f5fb9e094ed1458d599df2b2dc257cbdc6014d48197ba7f672ad9c1e48923286" Jul 14 19:49:12.662841 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.662805 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-89kkw"] Jul 14 19:49:12.726008 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.725963 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-89kkw"] Jul 14 19:49:12.726184 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.726082 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-89kkw" Jul 14 19:49:12.729511 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.729478 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-flz7p\"" Jul 14 19:49:12.866091 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.866055 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hb96p"] Jul 14 19:49:12.869593 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.869575 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hb96p" Jul 14 19:49:12.877896 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.877867 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hb96p"] Jul 14 19:49:12.885044 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.885016 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mmmd\" (UniqueName: \"kubernetes.io/projected/9937ec6d-89dd-4f86-af1e-5beadee4a986-kube-api-access-5mmmd\") pod \"authorino-58cd955675-89kkw\" (UID: \"9937ec6d-89dd-4f86-af1e-5beadee4a986\") " pod="kuadrant-system/authorino-58cd955675-89kkw" Jul 14 19:49:12.986180 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.986084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mmmd\" (UniqueName: \"kubernetes.io/projected/9937ec6d-89dd-4f86-af1e-5beadee4a986-kube-api-access-5mmmd\") pod \"authorino-58cd955675-89kkw\" (UID: \"9937ec6d-89dd-4f86-af1e-5beadee4a986\") " pod="kuadrant-system/authorino-58cd955675-89kkw" Jul 14 19:49:12.986180 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.986127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6n2s\" (UniqueName: \"kubernetes.io/projected/51f25d69-3898-4cad-ae7b-02cab6b942d6-kube-api-access-j6n2s\") pod \"authorino-75d556bd77-hb96p\" (UID: \"51f25d69-3898-4cad-ae7b-02cab6b942d6\") " pod="kuadrant-system/authorino-75d556bd77-hb96p" Jul 14 19:49:12.996716 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:12.996679 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mmmd\" (UniqueName: \"kubernetes.io/projected/9937ec6d-89dd-4f86-af1e-5beadee4a986-kube-api-access-5mmmd\") pod \"authorino-58cd955675-89kkw\" (UID: \"9937ec6d-89dd-4f86-af1e-5beadee4a986\") " pod="kuadrant-system/authorino-58cd955675-89kkw" Jul 14 19:49:13.035573 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.035528 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-89kkw" Jul 14 19:49:13.087517 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.087491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6n2s\" (UniqueName: \"kubernetes.io/projected/51f25d69-3898-4cad-ae7b-02cab6b942d6-kube-api-access-j6n2s\") pod \"authorino-75d556bd77-hb96p\" (UID: \"51f25d69-3898-4cad-ae7b-02cab6b942d6\") " pod="kuadrant-system/authorino-75d556bd77-hb96p" Jul 14 19:49:13.097321 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.097282 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6n2s\" (UniqueName: \"kubernetes.io/projected/51f25d69-3898-4cad-ae7b-02cab6b942d6-kube-api-access-j6n2s\") pod \"authorino-75d556bd77-hb96p\" (UID: \"51f25d69-3898-4cad-ae7b-02cab6b942d6\") " pod="kuadrant-system/authorino-75d556bd77-hb96p" Jul 14 19:49:13.171046 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.171018 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-89kkw"] Jul 14 19:49:13.173472 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:49:13.173442 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9937ec6d_89dd_4f86_af1e_5beadee4a986.slice/crio-300df5454260533d67075ed0cc959c657b5fef7124730dc2c59d4b2554713e0e WatchSource:0}: Error finding container 300df5454260533d67075ed0cc959c657b5fef7124730dc2c59d4b2554713e0e: Status 404 returned error can't find the container with id 300df5454260533d67075ed0cc959c657b5fef7124730dc2c59d4b2554713e0e Jul 14 19:49:13.174817 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.174792 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 19:49:13.179507 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.179482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hb96p" Jul 14 19:49:13.307920 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.307893 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hb96p"] Jul 14 19:49:13.310266 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:49:13.310239 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51f25d69_3898_4cad_ae7b_02cab6b942d6.slice/crio-b17e5de44f7fbe9939003c92b43a52f269752e73317666f716297417b0852017 WatchSource:0}: Error finding container b17e5de44f7fbe9939003c92b43a52f269752e73317666f716297417b0852017: Status 404 returned error can't find the container with id b17e5de44f7fbe9939003c92b43a52f269752e73317666f716297417b0852017 Jul 14 19:49:13.661214 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.661176 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-89kkw" event={"ID":"9937ec6d-89dd-4f86-af1e-5beadee4a986","Type":"ContainerStarted","Data":"300df5454260533d67075ed0cc959c657b5fef7124730dc2c59d4b2554713e0e"} Jul 14 19:49:13.662290 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:13.662264 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hb96p" event={"ID":"51f25d69-3898-4cad-ae7b-02cab6b942d6","Type":"ContainerStarted","Data":"b17e5de44f7fbe9939003c92b43a52f269752e73317666f716297417b0852017"} Jul 14 19:49:17.686635 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:17.686596 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-89kkw" event={"ID":"9937ec6d-89dd-4f86-af1e-5beadee4a986","Type":"ContainerStarted","Data":"38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af"} Jul 14 19:49:17.687967 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:17.687944 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hb96p" event={"ID":"51f25d69-3898-4cad-ae7b-02cab6b942d6","Type":"ContainerStarted","Data":"b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647"} Jul 14 19:49:17.701226 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:17.701171 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-89kkw" podStartSLOduration=1.643611081 podStartE2EDuration="5.701155294s" totalImagesPullingTime="4.057544213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:49:17.701103475 +0000 UTC m=+681.175202214" watchObservedRunningTime="2026-07-14 19:49:17.701155294 +0000 UTC m=+681.175254033" Jul 14 19:49:17.716842 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:17.716782 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hb96p" podStartSLOduration=1.782372308 podStartE2EDuration="5.716763588s" totalImagesPullingTime="3.93439128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:49:17.715189707 +0000 UTC m=+681.189288449" watchObservedRunningTime="2026-07-14 19:49:17.716763588 +0000 UTC m=+681.190862327" Jul 14 19:49:17.746342 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:17.746305 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-89kkw"] Jul 14 19:49:19.697200 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:19.697159 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-89kkw" podUID="9937ec6d-89dd-4f86-af1e-5beadee4a986" containerName="authorino" containerID="cri-o://38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af" gracePeriod=30 Jul 14 19:49:19.941949 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:19.941920 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-89kkw" Jul 14 19:49:20.049905 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.049866 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mmmd\" (UniqueName: \"kubernetes.io/projected/9937ec6d-89dd-4f86-af1e-5beadee4a986-kube-api-access-5mmmd\") pod \"9937ec6d-89dd-4f86-af1e-5beadee4a986\" (UID: \"9937ec6d-89dd-4f86-af1e-5beadee4a986\") " Jul 14 19:49:20.052078 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.052045 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9937ec6d-89dd-4f86-af1e-5beadee4a986-kube-api-access-5mmmd" (OuterVolumeSpecName: "kube-api-access-5mmmd") pod "9937ec6d-89dd-4f86-af1e-5beadee4a986" (UID: "9937ec6d-89dd-4f86-af1e-5beadee4a986"). InnerVolumeSpecName "kube-api-access-5mmmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:49:20.150592 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.150551 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mmmd\" (UniqueName: \"kubernetes.io/projected/9937ec6d-89dd-4f86-af1e-5beadee4a986-kube-api-access-5mmmd\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:49:20.703088 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.703057 2568 generic.go:356] "Generic (PLEG): container finished" podID="9937ec6d-89dd-4f86-af1e-5beadee4a986" containerID="38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af" exitCode=0 Jul 14 19:49:20.703575 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.703099 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-89kkw" Jul 14 19:49:20.703575 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.703121 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-89kkw" event={"ID":"9937ec6d-89dd-4f86-af1e-5beadee4a986","Type":"ContainerDied","Data":"38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af"} Jul 14 19:49:20.703575 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.703156 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-89kkw" event={"ID":"9937ec6d-89dd-4f86-af1e-5beadee4a986","Type":"ContainerDied","Data":"300df5454260533d67075ed0cc959c657b5fef7124730dc2c59d4b2554713e0e"} Jul 14 19:49:20.703575 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.703176 2568 scope.go:117] "RemoveContainer" containerID="38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af" Jul 14 19:49:20.711587 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.711566 2568 scope.go:117] "RemoveContainer" containerID="38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af" Jul 14 19:49:20.711863 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:49:20.711843 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af\": container with ID starting with 38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af not found: ID does not exist" containerID="38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af" Jul 14 19:49:20.711937 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.711871 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af"} err="failed to get container status \"38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af\": rpc error: code = NotFound desc = could not find container \"38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af\": container with ID starting with 38540776b3ecbd72e7bd38aa0f24995fa6c2250500a0ff9203af2dc6e05c45af not found: ID does not exist" Jul 14 19:49:20.725861 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.725821 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-89kkw"] Jul 14 19:49:20.732291 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:20.732258 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-89kkw"] Jul 14 19:49:21.045808 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:21.045773 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9937ec6d-89dd-4f86-af1e-5beadee4a986" path="/var/lib/kubelet/pods/9937ec6d-89dd-4f86-af1e-5beadee4a986/volumes" Jul 14 19:49:44.560688 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.560643 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-79dbh"] Jul 14 19:49:44.561300 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.561111 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9937ec6d-89dd-4f86-af1e-5beadee4a986" containerName="authorino" Jul 14 19:49:44.561300 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.561132 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9937ec6d-89dd-4f86-af1e-5beadee4a986" containerName="authorino" Jul 14 19:49:44.561300 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.561253 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9937ec6d-89dd-4f86-af1e-5beadee4a986" containerName="authorino" Jul 14 19:49:44.564459 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.564434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-79dbh" Jul 14 19:49:44.572102 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.572074 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-79dbh"] Jul 14 19:49:44.652482 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.652441 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jhhr\" (UniqueName: \"kubernetes.io/projected/d4982ee4-1605-41db-8418-12b202eadbd4-kube-api-access-7jhhr\") pod \"authorino-5d4b67c446-79dbh\" (UID: \"d4982ee4-1605-41db-8418-12b202eadbd4\") " pod="kuadrant-system/authorino-5d4b67c446-79dbh" Jul 14 19:49:44.753657 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.753616 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jhhr\" (UniqueName: \"kubernetes.io/projected/d4982ee4-1605-41db-8418-12b202eadbd4-kube-api-access-7jhhr\") pod \"authorino-5d4b67c446-79dbh\" (UID: \"d4982ee4-1605-41db-8418-12b202eadbd4\") " pod="kuadrant-system/authorino-5d4b67c446-79dbh" Jul 14 19:49:44.764317 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.764284 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jhhr\" (UniqueName: \"kubernetes.io/projected/d4982ee4-1605-41db-8418-12b202eadbd4-kube-api-access-7jhhr\") pod \"authorino-5d4b67c446-79dbh\" (UID: \"d4982ee4-1605-41db-8418-12b202eadbd4\") " pod="kuadrant-system/authorino-5d4b67c446-79dbh" Jul 14 19:49:44.787777 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.787744 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-79dbh"] Jul 14 19:49:44.788014 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.788001 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-79dbh" Jul 14 19:49:44.817261 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.817180 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-9fzs4"] Jul 14 19:49:44.820670 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.820648 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:44.823404 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.823365 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 14 19:49:44.826296 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.826270 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-9fzs4"] Jul 14 19:49:44.857481 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.857448 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-9fzs4"] Jul 14 19:49:44.857728 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:49:44.857702 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gjtmx tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-gjtmx tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-9fzs4" podUID="165bd42c-9ce9-4518-abf6-dc62822c3ea4" Jul 14 19:49:44.885843 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.885809 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c49c7648d-zh6qb"] Jul 14 19:49:44.889646 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.889626 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:49:44.897887 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.897827 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c49c7648d-zh6qb"] Jul 14 19:49:44.938534 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.938505 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-79dbh"] Jul 14 19:49:44.941092 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:49:44.941055 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4982ee4_1605_41db_8418_12b202eadbd4.slice/crio-dbc05aeacb40e4d926f45ea3d55597ff254cd344fee004ee89eb61e2c2672980 WatchSource:0}: Error finding container dbc05aeacb40e4d926f45ea3d55597ff254cd344fee004ee89eb61e2c2672980: Status 404 returned error can't find the container with id dbc05aeacb40e4d926f45ea3d55597ff254cd344fee004ee89eb61e2c2672980 Jul 14 19:49:44.956302 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.956272 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjtmx\" (UniqueName: \"kubernetes.io/projected/165bd42c-9ce9-4518-abf6-dc62822c3ea4-kube-api-access-gjtmx\") pod \"authorino-76877994-9fzs4\" (UID: \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\") " pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:44.956471 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.956329 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/165bd42c-9ce9-4518-abf6-dc62822c3ea4-tls-cert\") pod \"authorino-76877994-9fzs4\" (UID: \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\") " pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:44.956471 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.956416 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqlvh\" (UniqueName: \"kubernetes.io/projected/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-kube-api-access-kqlvh\") pod \"authorino-6c49c7648d-zh6qb\" (UID: \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\") " pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:49:44.956547 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:44.956484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-tls-cert\") pod \"authorino-6c49c7648d-zh6qb\" (UID: \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\") " pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:49:45.057657 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.057615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/165bd42c-9ce9-4518-abf6-dc62822c3ea4-tls-cert\") pod \"authorino-76877994-9fzs4\" (UID: \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\") " pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:45.057657 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.057663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqlvh\" (UniqueName: \"kubernetes.io/projected/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-kube-api-access-kqlvh\") pod \"authorino-6c49c7648d-zh6qb\" (UID: \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\") " pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:49:45.057926 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.057709 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-tls-cert\") pod \"authorino-6c49c7648d-zh6qb\" (UID: \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\") " pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:49:45.057926 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.057734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjtmx\" (UniqueName: \"kubernetes.io/projected/165bd42c-9ce9-4518-abf6-dc62822c3ea4-kube-api-access-gjtmx\") pod \"authorino-76877994-9fzs4\" (UID: \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\") " pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:45.060277 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.060253 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/165bd42c-9ce9-4518-abf6-dc62822c3ea4-tls-cert\") pod \"authorino-76877994-9fzs4\" (UID: \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\") " pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:45.060401 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.060317 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-tls-cert\") pod \"authorino-6c49c7648d-zh6qb\" (UID: \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\") " pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:49:45.066071 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.066041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjtmx\" (UniqueName: \"kubernetes.io/projected/165bd42c-9ce9-4518-abf6-dc62822c3ea4-kube-api-access-gjtmx\") pod \"authorino-76877994-9fzs4\" (UID: \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\") " pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:45.066174 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.066156 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqlvh\" (UniqueName: \"kubernetes.io/projected/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-kube-api-access-kqlvh\") pod \"authorino-6c49c7648d-zh6qb\" (UID: \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\") " pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:49:45.199627 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.199534 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:49:45.329969 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.329947 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c49c7648d-zh6qb"] Jul 14 19:49:45.335675 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:49:45.335646 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bb9c8b8_3d0d_488c_a0e6_ae6bd40f19bb.slice/crio-7c457f4fc2e50a5a18799595a4a3ec14bad295b33470f0755400a84a9acc303c WatchSource:0}: Error finding container 7c457f4fc2e50a5a18799595a4a3ec14bad295b33470f0755400a84a9acc303c: Status 404 returned error can't find the container with id 7c457f4fc2e50a5a18799595a4a3ec14bad295b33470f0755400a84a9acc303c Jul 14 19:49:45.822739 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.822699 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" event={"ID":"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb","Type":"ContainerStarted","Data":"e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98"} Jul 14 19:49:45.823185 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.822751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" event={"ID":"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb","Type":"ContainerStarted","Data":"7c457f4fc2e50a5a18799595a4a3ec14bad295b33470f0755400a84a9acc303c"} Jul 14 19:49:45.824235 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.824203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-79dbh" event={"ID":"d4982ee4-1605-41db-8418-12b202eadbd4","Type":"ContainerStarted","Data":"a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb"} Jul 14 19:49:45.824235 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.824224 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-79dbh" podUID="d4982ee4-1605-41db-8418-12b202eadbd4" containerName="authorino" containerID="cri-o://a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb" gracePeriod=30 Jul 14 19:49:45.824434 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.824241 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-79dbh" event={"ID":"d4982ee4-1605-41db-8418-12b202eadbd4","Type":"ContainerStarted","Data":"dbc05aeacb40e4d926f45ea3d55597ff254cd344fee004ee89eb61e2c2672980"} Jul 14 19:49:45.824434 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.824244 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:45.831897 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.831877 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:45.840075 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.840031 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" podStartSLOduration=1.511950012 podStartE2EDuration="1.84001731s" totalImagesPullingTime="328.067298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:49:45.838723032 +0000 UTC m=+709.312821773" watchObservedRunningTime="2026-07-14 19:49:45.84001731 +0000 UTC m=+709.314116076" Jul 14 19:49:45.868185 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.868146 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hb96p"] Jul 14 19:49:45.868369 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.868347 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hb96p" podUID="51f25d69-3898-4cad-ae7b-02cab6b942d6" containerName="authorino" containerID="cri-o://b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647" gracePeriod=30 Jul 14 19:49:45.871073 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.871019 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-79dbh" podStartSLOduration=1.5100562709999998 podStartE2EDuration="1.870998186s" totalImagesPullingTime="360.941915ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:49:45.870071628 +0000 UTC m=+709.344170370" watchObservedRunningTime="2026-07-14 19:49:45.870998186 +0000 UTC m=+709.345096927" Jul 14 19:49:45.964923 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.964825 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjtmx\" (UniqueName: \"kubernetes.io/projected/165bd42c-9ce9-4518-abf6-dc62822c3ea4-kube-api-access-gjtmx\") pod \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\" (UID: \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\") " Jul 14 19:49:45.964923 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.964909 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/165bd42c-9ce9-4518-abf6-dc62822c3ea4-tls-cert\") pod \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\" (UID: \"165bd42c-9ce9-4518-abf6-dc62822c3ea4\") " Jul 14 19:49:45.967159 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.967124 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165bd42c-9ce9-4518-abf6-dc62822c3ea4-kube-api-access-gjtmx" (OuterVolumeSpecName: "kube-api-access-gjtmx") pod "165bd42c-9ce9-4518-abf6-dc62822c3ea4" (UID: "165bd42c-9ce9-4518-abf6-dc62822c3ea4"). InnerVolumeSpecName "kube-api-access-gjtmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:49:45.967286 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:45.967188 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165bd42c-9ce9-4518-abf6-dc62822c3ea4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "165bd42c-9ce9-4518-abf6-dc62822c3ea4" (UID: "165bd42c-9ce9-4518-abf6-dc62822c3ea4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:49:46.065771 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.065724 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjtmx\" (UniqueName: \"kubernetes.io/projected/165bd42c-9ce9-4518-abf6-dc62822c3ea4-kube-api-access-gjtmx\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:49:46.065771 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.065765 2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/165bd42c-9ce9-4518-abf6-dc62822c3ea4-tls-cert\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:49:46.103544 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.103513 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-79dbh" Jul 14 19:49:46.139012 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.138980 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hb96p" Jul 14 19:49:46.166126 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.166093 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jhhr\" (UniqueName: \"kubernetes.io/projected/d4982ee4-1605-41db-8418-12b202eadbd4-kube-api-access-7jhhr\") pod \"d4982ee4-1605-41db-8418-12b202eadbd4\" (UID: \"d4982ee4-1605-41db-8418-12b202eadbd4\") " Jul 14 19:49:46.168255 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.168225 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4982ee4-1605-41db-8418-12b202eadbd4-kube-api-access-7jhhr" (OuterVolumeSpecName: "kube-api-access-7jhhr") pod "d4982ee4-1605-41db-8418-12b202eadbd4" (UID: "d4982ee4-1605-41db-8418-12b202eadbd4"). InnerVolumeSpecName "kube-api-access-7jhhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:49:46.267059 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.266950 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6n2s\" (UniqueName: \"kubernetes.io/projected/51f25d69-3898-4cad-ae7b-02cab6b942d6-kube-api-access-j6n2s\") pod \"51f25d69-3898-4cad-ae7b-02cab6b942d6\" (UID: \"51f25d69-3898-4cad-ae7b-02cab6b942d6\") " Jul 14 19:49:46.267244 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.267162 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jhhr\" (UniqueName: \"kubernetes.io/projected/d4982ee4-1605-41db-8418-12b202eadbd4-kube-api-access-7jhhr\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:49:46.269066 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.269036 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51f25d69-3898-4cad-ae7b-02cab6b942d6-kube-api-access-j6n2s" (OuterVolumeSpecName: "kube-api-access-j6n2s") pod "51f25d69-3898-4cad-ae7b-02cab6b942d6" (UID: "51f25d69-3898-4cad-ae7b-02cab6b942d6"). InnerVolumeSpecName "kube-api-access-j6n2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:49:46.368555 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.368515 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6n2s\" (UniqueName: \"kubernetes.io/projected/51f25d69-3898-4cad-ae7b-02cab6b942d6-kube-api-access-j6n2s\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:49:46.830609 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.830576 2568 generic.go:356] "Generic (PLEG): container finished" podID="d4982ee4-1605-41db-8418-12b202eadbd4" containerID="a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb" exitCode=0 Jul 14 19:49:46.831081 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.830620 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-79dbh" Jul 14 19:49:46.831081 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.830662 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-79dbh" event={"ID":"d4982ee4-1605-41db-8418-12b202eadbd4","Type":"ContainerDied","Data":"a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb"} Jul 14 19:49:46.831081 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.830706 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-79dbh" event={"ID":"d4982ee4-1605-41db-8418-12b202eadbd4","Type":"ContainerDied","Data":"dbc05aeacb40e4d926f45ea3d55597ff254cd344fee004ee89eb61e2c2672980"} Jul 14 19:49:46.831081 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.830725 2568 scope.go:117] "RemoveContainer" containerID="a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb" Jul 14 19:49:46.832065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.831956 2568 generic.go:356] "Generic (PLEG): container finished" podID="51f25d69-3898-4cad-ae7b-02cab6b942d6" containerID="b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647" exitCode=0 Jul 14 19:49:46.832065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.831999 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hb96p" Jul 14 19:49:46.832065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.832034 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hb96p" event={"ID":"51f25d69-3898-4cad-ae7b-02cab6b942d6","Type":"ContainerDied","Data":"b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647"} Jul 14 19:49:46.832065 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.832062 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hb96p" event={"ID":"51f25d69-3898-4cad-ae7b-02cab6b942d6","Type":"ContainerDied","Data":"b17e5de44f7fbe9939003c92b43a52f269752e73317666f716297417b0852017"} Jul 14 19:49:46.832326 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.832310 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9fzs4" Jul 14 19:49:46.841252 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.841224 2568 scope.go:117] "RemoveContainer" containerID="a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb" Jul 14 19:49:46.841631 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:49:46.841605 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb\": container with ID starting with a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb not found: ID does not exist" containerID="a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb" Jul 14 19:49:46.841724 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.841643 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb"} err="failed to get container status \"a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb\": rpc error: code = NotFound desc = could not find container \"a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb\": container with ID starting with a2de2b893a29ba137edda9fc708fb52ea43fa32c92f4002027d256b4473ed7fb not found: ID does not exist" Jul 14 19:49:46.841724 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.841670 2568 scope.go:117] "RemoveContainer" containerID="b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647" Jul 14 19:49:46.849783 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.849766 2568 scope.go:117] "RemoveContainer" containerID="b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647" Jul 14 19:49:46.850074 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:49:46.850053 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647\": container with ID starting with b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647 not found: ID does not exist" containerID="b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647" Jul 14 19:49:46.850119 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.850083 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647"} err="failed to get container status \"b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647\": rpc error: code = NotFound desc = could not find container \"b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647\": container with ID starting with b52d5f356a938a03a2ee06fd988757796442eb3bd5ff9a3999f2e10797829647 not found: ID does not exist" Jul 14 19:49:46.859203 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.859164 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-79dbh"] Jul 14 19:49:46.864145 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.864117 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-79dbh"] Jul 14 19:49:46.891848 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.891763 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-9fzs4"] Jul 14 19:49:46.898474 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.898448 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-9fzs4"] Jul 14 19:49:46.914086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.910441 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hb96p"] Jul 14 19:49:46.916756 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:46.916725 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hb96p"] Jul 14 19:49:47.045946 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:47.045913 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="165bd42c-9ce9-4518-abf6-dc62822c3ea4" path="/var/lib/kubelet/pods/165bd42c-9ce9-4518-abf6-dc62822c3ea4/volumes" Jul 14 19:49:47.046161 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:47.046149 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51f25d69-3898-4cad-ae7b-02cab6b942d6" path="/var/lib/kubelet/pods/51f25d69-3898-4cad-ae7b-02cab6b942d6/volumes" Jul 14 19:49:47.046456 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:49:47.046444 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4982ee4-1605-41db-8418-12b202eadbd4" path="/var/lib/kubelet/pods/d4982ee4-1605-41db-8418-12b202eadbd4/volumes" Jul 14 19:50:27.732804 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.732758 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s"] Jul 14 19:50:27.733436 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.733246 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4982ee4-1605-41db-8418-12b202eadbd4" containerName="authorino" Jul 14 19:50:27.733436 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.733268 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4982ee4-1605-41db-8418-12b202eadbd4" containerName="authorino" Jul 14 19:50:27.733436 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.733295 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51f25d69-3898-4cad-ae7b-02cab6b942d6" containerName="authorino" Jul 14 19:50:27.733436 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.733304 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="51f25d69-3898-4cad-ae7b-02cab6b942d6" containerName="authorino" Jul 14 19:50:27.737404 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.733797 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4982ee4-1605-41db-8418-12b202eadbd4" containerName="authorino" Jul 14 19:50:27.737404 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.733831 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="51f25d69-3898-4cad-ae7b-02cab6b942d6" containerName="authorino" Jul 14 19:50:27.737832 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.737799 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:27.741835 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.741810 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 14 19:50:27.741991 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.741810 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pxtt6\"" Jul 14 19:50:27.748754 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.748729 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s"] Jul 14 19:50:27.811780 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.811738 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjd55\" (UniqueName: \"kubernetes.io/projected/1eaa23e0-ca9a-4a2f-a787-c69ffd77a282-kube-api-access-mjd55\") pod \"payload-pre-processing-74584bb9f8-wbk6s\" (UID: \"1eaa23e0-ca9a-4a2f-a787-c69ffd77a282\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:27.811969 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.811880 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1eaa23e0-ca9a-4a2f-a787-c69ffd77a282-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-wbk6s\" (UID: \"1eaa23e0-ca9a-4a2f-a787-c69ffd77a282\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:27.913303 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.913268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1eaa23e0-ca9a-4a2f-a787-c69ffd77a282-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-wbk6s\" (UID: \"1eaa23e0-ca9a-4a2f-a787-c69ffd77a282\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:27.913504 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.913326 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjd55\" (UniqueName: \"kubernetes.io/projected/1eaa23e0-ca9a-4a2f-a787-c69ffd77a282-kube-api-access-mjd55\") pod \"payload-pre-processing-74584bb9f8-wbk6s\" (UID: \"1eaa23e0-ca9a-4a2f-a787-c69ffd77a282\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:27.914011 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.913990 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1eaa23e0-ca9a-4a2f-a787-c69ffd77a282-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-wbk6s\" (UID: \"1eaa23e0-ca9a-4a2f-a787-c69ffd77a282\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:27.927574 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:27.927538 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjd55\" (UniqueName: \"kubernetes.io/projected/1eaa23e0-ca9a-4a2f-a787-c69ffd77a282-kube-api-access-mjd55\") pod \"payload-pre-processing-74584bb9f8-wbk6s\" (UID: \"1eaa23e0-ca9a-4a2f-a787-c69ffd77a282\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:28.049711 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:28.049665 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:28.193967 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:28.193937 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s"] Jul 14 19:50:28.196949 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:50:28.196918 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eaa23e0_ca9a_4a2f_a787_c69ffd77a282.slice/crio-4f39ec196fd7d73fc80e083471fd1a9ef8ed9413a042a8c29e34f81647ba7017 WatchSource:0}: Error finding container 4f39ec196fd7d73fc80e083471fd1a9ef8ed9413a042a8c29e34f81647ba7017: Status 404 returned error can't find the container with id 4f39ec196fd7d73fc80e083471fd1a9ef8ed9413a042a8c29e34f81647ba7017 Jul 14 19:50:29.024220 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:29.024182 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" event={"ID":"1eaa23e0-ca9a-4a2f-a787-c69ffd77a282","Type":"ContainerStarted","Data":"4f39ec196fd7d73fc80e083471fd1a9ef8ed9413a042a8c29e34f81647ba7017"} Jul 14 19:50:31.034045 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:31.034004 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" event={"ID":"1eaa23e0-ca9a-4a2f-a787-c69ffd77a282","Type":"ContainerStarted","Data":"18588fbb5643e2ab2be00aa9aa6e701df1ef64803546000f28b8b1b55c61ec1d"} Jul 14 19:50:31.034472 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:31.034129 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:31.053259 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:31.053195 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" podStartSLOduration=2.236859588 podStartE2EDuration="4.053172892s" totalImagesPullingTime="1.816313304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:50:31.052759671 +0000 UTC m=+754.526858446" watchObservedRunningTime="2026-07-14 19:50:31.053172892 +0000 UTC m=+754.527271633" Jul 14 19:50:42.040507 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:42.040476 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wbk6s" Jul 14 19:50:55.128641 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.128602 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8"] Jul 14 19:50:55.131902 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.131881 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.134847 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.134820 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 19:50:55.134986 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.134914 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c2bvl\"" Jul 14 19:50:55.135180 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.135164 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 19:50:55.136023 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.136007 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 19:50:55.144589 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.144564 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8"] Jul 14 19:50:55.260487 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.260448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.260667 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.260510 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.260667 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.260553 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.260667 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.260573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfx2s\" (UniqueName: \"kubernetes.io/projected/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-kube-api-access-dfx2s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.260667 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.260630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.260804 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.260702 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.361919 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.361877 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.362086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.361930 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.362086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.361954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.362086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.361977 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfx2s\" (UniqueName: \"kubernetes.io/projected/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-kube-api-access-dfx2s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.362086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.362010 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.362086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.362052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.362404 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.362357 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.362463 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.362443 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.362500 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.362486 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.364417 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.364396 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.364611 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.364594 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.371554 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.371520 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfx2s\" (UniqueName: \"kubernetes.io/projected/1bfdc529-c9f1-4cca-9c5b-513e578e7bc4-kube-api-access-dfx2s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w4vj8\" (UID: \"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.442415 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.442283 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:50:55.576215 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:55.576184 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8"] Jul 14 19:50:55.578395 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:50:55.578353 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bfdc529_c9f1_4cca_9c5b_513e578e7bc4.slice/crio-3931d9fffce2c41ca3c183711b05909b0a5b017c6fd5ad9083366e82de206678 WatchSource:0}: Error finding container 3931d9fffce2c41ca3c183711b05909b0a5b017c6fd5ad9083366e82de206678: Status 404 returned error can't find the container with id 3931d9fffce2c41ca3c183711b05909b0a5b017c6fd5ad9083366e82de206678 Jul 14 19:50:56.156856 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:56.156815 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" event={"ID":"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4","Type":"ContainerStarted","Data":"3931d9fffce2c41ca3c183711b05909b0a5b017c6fd5ad9083366e82de206678"} Jul 14 19:50:57.162683 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:57.162643 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" event={"ID":"1bfdc529-c9f1-4cca-9c5b-513e578e7bc4","Type":"ContainerStarted","Data":"27f234e3cb5b13fa3145eb43d8dd72e5910b6b6a6a53fbacea623987070fc043"} Jul 14 19:50:57.188366 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:50:57.188304 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" podStartSLOduration=1.1021722999999999 podStartE2EDuration="2.188289631s" totalImagesPullingTime="1.086117331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:50:57.18627117 +0000 UTC m=+780.660369909" watchObservedRunningTime="2026-07-14 19:50:57.188289631 +0000 UTC m=+780.662388436" Jul 14 19:51:03.046701 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.046672 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9"] Jul 14 19:51:03.055785 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.055751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.059553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.059521 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 19:51:03.066237 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.066208 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9"] Jul 14 19:51:03.232998 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.232954 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.233222 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.233019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.233222 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.233040 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.233222 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.233093 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m5xz\" (UniqueName: \"kubernetes.io/projected/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-kube-api-access-6m5xz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.233222 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.233132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.233372 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.233247 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334495 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334359 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334495 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334495 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334737 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334501 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334737 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m5xz\" (UniqueName: \"kubernetes.io/projected/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-kube-api-access-6m5xz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334737 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334860 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334834 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334900 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334850 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.334934 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.334917 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.336871 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.336837 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.337107 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.337086 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.344364 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.344326 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m5xz\" (UniqueName: \"kubernetes.io/projected/5faa70f5-fafa-4375-a7c1-6a5f01b7c493-kube-api-access-6m5xz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jqzr9\" (UID: \"5faa70f5-fafa-4375-a7c1-6a5f01b7c493\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.366915 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.366877 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:03.504759 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:03.504722 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9"] Jul 14 19:51:03.508201 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:51:03.508170 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5faa70f5_fafa_4375_a7c1_6a5f01b7c493.slice/crio-473368a8cc2c96a3c72c8105014d54cee5ef722d44a38380145e75fb8b619288 WatchSource:0}: Error finding container 473368a8cc2c96a3c72c8105014d54cee5ef722d44a38380145e75fb8b619288: Status 404 returned error can't find the container with id 473368a8cc2c96a3c72c8105014d54cee5ef722d44a38380145e75fb8b619288 Jul 14 19:51:04.195148 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:04.195060 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" event={"ID":"5faa70f5-fafa-4375-a7c1-6a5f01b7c493","Type":"ContainerStarted","Data":"67dae58d1900e9f681d1af0bd34ff0d2ebf5b9de1b7b0593195a59b491c9852b"} Jul 14 19:51:04.195148 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:04.195116 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" event={"ID":"5faa70f5-fafa-4375-a7c1-6a5f01b7c493","Type":"ContainerStarted","Data":"473368a8cc2c96a3c72c8105014d54cee5ef722d44a38380145e75fb8b619288"} Jul 14 19:51:04.234323 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:04.234261 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" podStartSLOduration=0.888531364 podStartE2EDuration="1.234241319s" totalImagesPullingTime="345.709955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:51:04.2329048 +0000 UTC m=+787.707003539" watchObservedRunningTime="2026-07-14 19:51:04.234241319 +0000 UTC m=+787.708340056" Jul 14 19:51:05.443106 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:05.443069 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:51:05.443570 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:05.443129 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:51:05.455375 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:05.455346 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:51:06.218079 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:06.218049 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w4vj8" Jul 14 19:51:13.367874 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:13.367834 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:13.368345 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:13.367899 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:13.380463 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:13.380423 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:14.254160 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:14.254129 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jqzr9" Jul 14 19:51:15.827854 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.827822 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w"] Jul 14 19:51:15.832008 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.831989 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.835305 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.835281 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 19:51:15.842245 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.842216 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w"] Jul 14 19:51:15.846855 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.846828 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28z2w\" (UniqueName: \"kubernetes.io/projected/8c3c3496-16d3-4e6b-906f-a61e3d543fad-kube-api-access-28z2w\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.846991 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.846865 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.846991 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.846883 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.846991 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.846903 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.846991 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.846920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c3c3496-16d3-4e6b-906f-a61e3d543fad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.846991 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.846940 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.947404 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.947330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28z2w\" (UniqueName: \"kubernetes.io/projected/8c3c3496-16d3-4e6b-906f-a61e3d543fad-kube-api-access-28z2w\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.947601 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.947424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.947601 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.947443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.947601 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.947474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.947601 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.947493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c3c3496-16d3-4e6b-906f-a61e3d543fad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.947601 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.947525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.947950 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.947919 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.948160 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.947995 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.948266 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.948060 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.949805 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.949780 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c3c3496-16d3-4e6b-906f-a61e3d543fad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.950251 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.950229 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c3c3496-16d3-4e6b-906f-a61e3d543fad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:15.956392 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:15.956348 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28z2w\" (UniqueName: \"kubernetes.io/projected/8c3c3496-16d3-4e6b-906f-a61e3d543fad-kube-api-access-28z2w\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w\" (UID: \"8c3c3496-16d3-4e6b-906f-a61e3d543fad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:16.143969 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:16.143871 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:16.286086 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:16.286060 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w"] Jul 14 19:51:16.288219 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:51:16.288178 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c3c3496_16d3_4e6b_906f_a61e3d543fad.slice/crio-3d19d46dcd00be01598833d3a27ae4d33f59446079f0272868c650c184871e0d WatchSource:0}: Error finding container 3d19d46dcd00be01598833d3a27ae4d33f59446079f0272868c650c184871e0d: Status 404 returned error can't find the container with id 3d19d46dcd00be01598833d3a27ae4d33f59446079f0272868c650c184871e0d Jul 14 19:51:17.259058 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:17.259018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" event={"ID":"8c3c3496-16d3-4e6b-906f-a61e3d543fad","Type":"ContainerStarted","Data":"1c10874020562b871ed8e0162d2e208a6956c0be666a7b108b949facc5b1bcd6"} Jul 14 19:51:17.259494 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:17.259073 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" event={"ID":"8c3c3496-16d3-4e6b-906f-a61e3d543fad","Type":"ContainerStarted","Data":"3d19d46dcd00be01598833d3a27ae4d33f59446079f0272868c650c184871e0d"} Jul 14 19:51:17.298287 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:17.298234 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" podStartSLOduration=2.100818953 podStartE2EDuration="2.298199827s" totalImagesPullingTime="197.380874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:51:17.295558139 +0000 UTC m=+800.769656880" watchObservedRunningTime="2026-07-14 19:51:17.298199827 +0000 UTC m=+800.772298566" Jul 14 19:51:26.144563 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:26.144529 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:26.144998 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:26.144590 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:26.157079 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:26.157053 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:26.313724 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:26.313690 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w" Jul 14 19:51:28.535782 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.535749 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps"] Jul 14 19:51:28.539179 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.539160 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.542883 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.542858 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 14 19:51:28.566824 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.566790 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps"] Jul 14 19:51:28.661855 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.661821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krbrj\" (UniqueName: \"kubernetes.io/projected/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-kube-api-access-krbrj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.662037 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.661867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.662037 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.661929 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.662037 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.661998 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.662037 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.662032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.662175 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.662070 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.763587 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.763544 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.763587 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.763595 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.763846 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.763613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.763846 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.763728 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.763951 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.763845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krbrj\" (UniqueName: \"kubernetes.io/projected/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-kube-api-access-krbrj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.763951 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.763886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.764110 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.764082 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.764164 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.764095 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.764202 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.764163 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.765928 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.765906 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.766184 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.766168 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.772720 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.772694 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krbrj\" (UniqueName: \"kubernetes.io/projected/c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f-kube-api-access-krbrj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p5hps\" (UID: \"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:28.850179 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:28.850087 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:29.002021 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:29.001993 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps"] Jul 14 19:51:29.004202 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:51:29.004176 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b5b04a_fd1a_47c2_a157_1d0abfa48b5f.slice/crio-6df67d16295bdbc96547cca5b98127c58afa03f179d0cc0df32805ee6fa0f349 WatchSource:0}: Error finding container 6df67d16295bdbc96547cca5b98127c58afa03f179d0cc0df32805ee6fa0f349: Status 404 returned error can't find the container with id 6df67d16295bdbc96547cca5b98127c58afa03f179d0cc0df32805ee6fa0f349 Jul 14 19:51:29.316960 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:29.316927 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" event={"ID":"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f","Type":"ContainerStarted","Data":"88f23871d28314b0330012ee854f637081c8f1863ae048f266a2d36ff3f7d2c8"} Jul 14 19:51:29.317136 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:29.316988 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" event={"ID":"c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f","Type":"ContainerStarted","Data":"6df67d16295bdbc96547cca5b98127c58afa03f179d0cc0df32805ee6fa0f349"} Jul 14 19:51:29.341475 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:29.341414 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" podStartSLOduration=1.120462518 podStartE2EDuration="1.34137126s" totalImagesPullingTime="220.908742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:51:29.338506377 +0000 UTC m=+812.812605117" watchObservedRunningTime="2026-07-14 19:51:29.34137126 +0000 UTC m=+812.815470041" Jul 14 19:51:35.326651 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.326615 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp"] Jul 14 19:51:35.331627 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.331608 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.334206 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.334182 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 14 19:51:35.341708 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.341681 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp"] Jul 14 19:51:35.428122 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.428061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.428122 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.428132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.428357 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.428237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.428357 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.428285 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.428357 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.428342 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-577h5\" (UniqueName: \"kubernetes.io/projected/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-kube-api-access-577h5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.428515 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.428443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.529802 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.529760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.530000 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.529815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.530000 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.529863 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.530000 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.529932 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.530000 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.529972 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.530203 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.530006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-577h5\" (UniqueName: \"kubernetes.io/projected/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-kube-api-access-577h5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.530296 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.530272 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.530353 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.530307 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.530455 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.530359 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.532306 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.532280 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.532599 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.532579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.539140 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.539109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-577h5\" (UniqueName: \"kubernetes.io/projected/937c5a94-eebe-4c07-82c3-4fee2a7f55bd-kube-api-access-577h5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp\" (UID: \"937c5a94-eebe-4c07-82c3-4fee2a7f55bd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.641594 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.641484 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:35.774555 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:35.774524 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp"] Jul 14 19:51:35.776669 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:51:35.776632 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod937c5a94_eebe_4c07_82c3_4fee2a7f55bd.slice/crio-10c0d3ff9c5eb41874d3ce55db52902d157299663cca718ce3216b8d1c936535 WatchSource:0}: Error finding container 10c0d3ff9c5eb41874d3ce55db52902d157299663cca718ce3216b8d1c936535: Status 404 returned error can't find the container with id 10c0d3ff9c5eb41874d3ce55db52902d157299663cca718ce3216b8d1c936535 Jul 14 19:51:36.351578 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:36.351544 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" event={"ID":"937c5a94-eebe-4c07-82c3-4fee2a7f55bd","Type":"ContainerStarted","Data":"80519a00636769bc51787e35d6b7f45cf735acfcecf5123ccdba155db39e4d2f"} Jul 14 19:51:36.351578 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:36.351585 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" event={"ID":"937c5a94-eebe-4c07-82c3-4fee2a7f55bd","Type":"ContainerStarted","Data":"10c0d3ff9c5eb41874d3ce55db52902d157299663cca718ce3216b8d1c936535"} Jul 14 19:51:36.375189 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:36.375125 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" podStartSLOduration=1.169799276 podStartE2EDuration="1.375106833s" totalImagesPullingTime="205.307557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:51:36.373044337 +0000 UTC m=+819.847143077" watchObservedRunningTime="2026-07-14 19:51:36.375106833 +0000 UTC m=+819.849205572" Jul 14 19:51:38.851159 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:38.851116 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:38.851644 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:38.851455 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:38.863762 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:38.863735 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:39.376863 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:39.376826 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p5hps" Jul 14 19:51:45.642186 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:45.642142 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:45.642728 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:45.642203 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:45.654802 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:45.654771 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:51:46.408446 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:51:46.408416 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp" Jul 14 19:52:35.082189 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.082151 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-797bcdc8bf-d27v4"] Jul 14 19:52:35.085509 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.085490 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-797bcdc8bf-d27v4" Jul 14 19:52:35.096428 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.096398 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-797bcdc8bf-d27v4"] Jul 14 19:52:35.276447 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.276404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn48t\" (UniqueName: \"kubernetes.io/projected/3a5843df-1ee6-4542-bd34-eeb629917806-kube-api-access-pn48t\") pod \"authorino-797bcdc8bf-d27v4\" (UID: \"3a5843df-1ee6-4542-bd34-eeb629917806\") " pod="kuadrant-system/authorino-797bcdc8bf-d27v4" Jul 14 19:52:35.276636 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.276530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a5843df-1ee6-4542-bd34-eeb629917806-tls-cert\") pod \"authorino-797bcdc8bf-d27v4\" (UID: \"3a5843df-1ee6-4542-bd34-eeb629917806\") " pod="kuadrant-system/authorino-797bcdc8bf-d27v4" Jul 14 19:52:35.377395 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.377276 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn48t\" (UniqueName: \"kubernetes.io/projected/3a5843df-1ee6-4542-bd34-eeb629917806-kube-api-access-pn48t\") pod \"authorino-797bcdc8bf-d27v4\" (UID: \"3a5843df-1ee6-4542-bd34-eeb629917806\") " pod="kuadrant-system/authorino-797bcdc8bf-d27v4" Jul 14 19:52:35.377553 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.377443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a5843df-1ee6-4542-bd34-eeb629917806-tls-cert\") pod \"authorino-797bcdc8bf-d27v4\" (UID: \"3a5843df-1ee6-4542-bd34-eeb629917806\") " pod="kuadrant-system/authorino-797bcdc8bf-d27v4" Jul 14 19:52:35.379999 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.379972 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a5843df-1ee6-4542-bd34-eeb629917806-tls-cert\") pod \"authorino-797bcdc8bf-d27v4\" (UID: \"3a5843df-1ee6-4542-bd34-eeb629917806\") " pod="kuadrant-system/authorino-797bcdc8bf-d27v4" Jul 14 19:52:35.385895 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.385870 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn48t\" (UniqueName: \"kubernetes.io/projected/3a5843df-1ee6-4542-bd34-eeb629917806-kube-api-access-pn48t\") pod \"authorino-797bcdc8bf-d27v4\" (UID: \"3a5843df-1ee6-4542-bd34-eeb629917806\") " pod="kuadrant-system/authorino-797bcdc8bf-d27v4" Jul 14 19:52:35.395572 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.395543 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-797bcdc8bf-d27v4" Jul 14 19:52:35.535906 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.535877 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-797bcdc8bf-d27v4"] Jul 14 19:52:35.537930 ip-10-0-128-126 kubenswrapper[2568]: W0714 19:52:35.537903 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a5843df_1ee6_4542_bd34_eeb629917806.slice/crio-47354848c81f08c28aead2b7eb0da5533869845f6bf38fd254f0638333896467 WatchSource:0}: Error finding container 47354848c81f08c28aead2b7eb0da5533869845f6bf38fd254f0638333896467: Status 404 returned error can't find the container with id 47354848c81f08c28aead2b7eb0da5533869845f6bf38fd254f0638333896467 Jul 14 19:52:35.632860 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:35.632770 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-797bcdc8bf-d27v4" event={"ID":"3a5843df-1ee6-4542-bd34-eeb629917806","Type":"ContainerStarted","Data":"47354848c81f08c28aead2b7eb0da5533869845f6bf38fd254f0638333896467"} Jul 14 19:52:36.639400 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:36.639224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-797bcdc8bf-d27v4" event={"ID":"3a5843df-1ee6-4542-bd34-eeb629917806","Type":"ContainerStarted","Data":"53d2dc26cd658b4a946c4ede381ca7281febb40a45789ac3096d4a36f9f48329"} Jul 14 19:52:36.657454 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:36.657369 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-797bcdc8bf-d27v4" podStartSLOduration=0.704324998 podStartE2EDuration="1.657349158s" totalImagesPullingTime="953.02416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:52:36.655926403 +0000 UTC m=+880.130025198" watchObservedRunningTime="2026-07-14 19:52:36.657349158 +0000 UTC m=+880.131447899" Jul 14 19:52:36.698540 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:36.698503 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c49c7648d-zh6qb"] Jul 14 19:52:36.698746 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:36.698718 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" podUID="0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb" containerName="authorino" containerID="cri-o://e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98" gracePeriod=30 Jul 14 19:52:36.957123 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:36.957096 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:52:36.992837 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:36.992737 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqlvh\" (UniqueName: \"kubernetes.io/projected/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-kube-api-access-kqlvh\") pod \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\" (UID: \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\") " Jul 14 19:52:36.992837 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:36.992830 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-tls-cert\") pod \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\" (UID: \"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb\") " Jul 14 19:52:36.995513 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:36.995467 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-kube-api-access-kqlvh" (OuterVolumeSpecName: "kube-api-access-kqlvh") pod "0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb" (UID: "0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb"). InnerVolumeSpecName "kube-api-access-kqlvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 19:52:37.003994 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.003961 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb" (UID: "0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 19:52:37.094316 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.094276 2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-tls-cert\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:52:37.094316 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.094313 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqlvh\" (UniqueName: \"kubernetes.io/projected/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb-kube-api-access-kqlvh\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 19:52:37.645413 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.645347 2568 generic.go:356] "Generic (PLEG): container finished" podID="0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb" containerID="e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98" exitCode=0 Jul 14 19:52:37.645413 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.645408 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" Jul 14 19:52:37.645943 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.645442 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" event={"ID":"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb","Type":"ContainerDied","Data":"e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98"} Jul 14 19:52:37.645943 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.645492 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c49c7648d-zh6qb" event={"ID":"0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb","Type":"ContainerDied","Data":"7c457f4fc2e50a5a18799595a4a3ec14bad295b33470f0755400a84a9acc303c"} Jul 14 19:52:37.645943 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.645520 2568 scope.go:117] "RemoveContainer" containerID="e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98" Jul 14 19:52:37.654809 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.654787 2568 scope.go:117] "RemoveContainer" containerID="e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98" Jul 14 19:52:37.655126 ip-10-0-128-126 kubenswrapper[2568]: E0714 19:52:37.655104 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98\": container with ID starting with e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98 not found: ID does not exist" containerID="e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98" Jul 14 19:52:37.655194 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.655138 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98"} err="failed to get container status \"e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98\": rpc error: code = NotFound desc = could not find container \"e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98\": container with ID starting with e7047313a291206ac8f2567e53906dfeef2e90c8941707910ce0de13bb997c98 not found: ID does not exist" Jul 14 19:52:37.667726 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.667689 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c49c7648d-zh6qb"] Jul 14 19:52:37.676184 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:37.676141 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c49c7648d-zh6qb"] Jul 14 19:52:39.046120 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:39.046081 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb" path="/var/lib/kubelet/pods/0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb/volumes" Jul 14 19:52:57.019468 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:57.019368 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:52:57.022394 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:52:57.022358 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:57:57.050593 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:57:57.050559 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 19:57:57.054762 ip-10-0-128-126 kubenswrapper[2568]: I0714 19:57:57.054735 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:02:57.079102 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:02:57.079067 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:02:57.084850 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:02:57.084825 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:07:57.107675 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:07:57.107643 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:07:57.114985 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:07:57.114960 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:12:57.143349 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:12:57.143316 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:12:57.149525 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:12:57.149501 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:17:57.171887 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:17:57.171852 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:17:57.178535 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:17:57.178511 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:22:22.287073 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.287040 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59"] Jul 14 20:22:22.287595 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.287499 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb" containerName="authorino" Jul 14 20:22:22.287595 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.287515 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb" containerName="authorino" Jul 14 20:22:22.287595 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.287591 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bb9c8b8-3d0d-488c-a0e6-ae6bd40f19bb" containerName="authorino" Jul 14 20:22:22.290560 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.290542 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.300961 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.300931 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"auth-test-model-ecc78d-kserve-self-signed-certs\"" Jul 14 20:22:22.301133 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.300931 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"default-dockercfg-gz8ps\"" Jul 14 20:22:22.301798 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.300988 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"kube-root-ca.crt\"" Jul 14 20:22:22.301798 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.301000 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"openshift-service-ca.crt\"" Jul 14 20:22:22.303363 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.303342 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59"] Jul 14 20:22:22.362203 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.362157 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-model-cache\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.362415 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.362293 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-dshm\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.362415 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.362316 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-tmp-dir\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.362415 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.362333 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mds2t\" (UniqueName: \"kubernetes.io/projected/7e32864a-58aa-493a-b5b2-b29303cb103f-kube-api-access-mds2t\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.362415 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.362357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e32864a-58aa-493a-b5b2-b29303cb103f-tls-certs\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.362559 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.362421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-home\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.463437 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463359 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-model-cache\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.463613 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-dshm\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.463613 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463567 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-tmp-dir\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.463613 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463590 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mds2t\" (UniqueName: \"kubernetes.io/projected/7e32864a-58aa-493a-b5b2-b29303cb103f-kube-api-access-mds2t\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.463773 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463621 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e32864a-58aa-493a-b5b2-b29303cb103f-tls-certs\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.463773 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463656 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-home\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.463870 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463838 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-model-cache\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.463986 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463961 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-tmp-dir\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.464043 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.463983 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-home\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.465861 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.465826 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-dshm\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.466073 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.466053 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e32864a-58aa-493a-b5b2-b29303cb103f-tls-certs\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.476624 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.476598 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mds2t\" (UniqueName: \"kubernetes.io/projected/7e32864a-58aa-493a-b5b2-b29303cb103f-kube-api-access-mds2t\") pod \"auth-test-model-ecc78d-kserve-5d75fbd555-75n59\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.600270 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.600178 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:22.760889 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:22.760863 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59"] Jul 14 20:22:22.763510 ip-10-0-128-126 kubenswrapper[2568]: W0714 20:22:22.763485 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e32864a_58aa_493a_b5b2_b29303cb103f.slice/crio-f77e3f8bd99d3f9e6d8a831e73099acfe7038f6be7006c124869d0378b9acfb3 WatchSource:0}: Error finding container f77e3f8bd99d3f9e6d8a831e73099acfe7038f6be7006c124869d0378b9acfb3: Status 404 returned error can't find the container with id f77e3f8bd99d3f9e6d8a831e73099acfe7038f6be7006c124869d0378b9acfb3 Jul 14 20:22:23.273588 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:23.273549 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" event={"ID":"7e32864a-58aa-493a-b5b2-b29303cb103f","Type":"ContainerStarted","Data":"289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922"} Jul 14 20:22:23.273588 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:23.273594 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" event={"ID":"7e32864a-58aa-493a-b5b2-b29303cb103f","Type":"ContainerStarted","Data":"f77e3f8bd99d3f9e6d8a831e73099acfe7038f6be7006c124869d0378b9acfb3"} Jul 14 20:22:23.319408 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:23.319331 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" podStartSLOduration=1.319314243 podStartE2EDuration="1.319314243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:22:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:23.318408673 +0000 UTC m=+2666.792507409" watchObservedRunningTime="2026-07-14 20:22:23.319314243 +0000 UTC m=+2666.793412982" Jul 14 20:22:32.600634 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:32.600593 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:32.601032 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:32.600656 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:32.613253 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:32.613225 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:33.333730 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:33.333694 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:22:57.200500 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:57.200467 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:22:57.207232 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:22:57.207204 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:24:27.213035 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:27.212992 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59"] Jul 14 20:24:42.233728 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.233683 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" podUID="7e32864a-58aa-493a-b5b2-b29303cb103f" containerName="main" containerID="cri-o://289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922" gracePeriod=45 Jul 14 20:24:42.479058 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.479032 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:24:42.585253 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585224 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-tmp-dir\") pod \"7e32864a-58aa-493a-b5b2-b29303cb103f\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " Jul 14 20:24:42.585447 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585300 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-model-cache\") pod \"7e32864a-58aa-493a-b5b2-b29303cb103f\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " Jul 14 20:24:42.585447 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585322 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mds2t\" (UniqueName: \"kubernetes.io/projected/7e32864a-58aa-493a-b5b2-b29303cb103f-kube-api-access-mds2t\") pod \"7e32864a-58aa-493a-b5b2-b29303cb103f\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " Jul 14 20:24:42.585447 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585346 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e32864a-58aa-493a-b5b2-b29303cb103f-tls-certs\") pod \"7e32864a-58aa-493a-b5b2-b29303cb103f\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " Jul 14 20:24:42.585447 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585361 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-dshm\") pod \"7e32864a-58aa-493a-b5b2-b29303cb103f\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " Jul 14 20:24:42.585447 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585403 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-home\") pod \"7e32864a-58aa-493a-b5b2-b29303cb103f\" (UID: \"7e32864a-58aa-493a-b5b2-b29303cb103f\") " Jul 14 20:24:42.585725 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585582 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e32864a-58aa-493a-b5b2-b29303cb103f" (UID: "7e32864a-58aa-493a-b5b2-b29303cb103f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:24:42.585725 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585654 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-tmp-dir\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:24:42.585840 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585707 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e32864a-58aa-493a-b5b2-b29303cb103f" (UID: "7e32864a-58aa-493a-b5b2-b29303cb103f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:24:42.585840 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.585766 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-home" (OuterVolumeSpecName: "home") pod "7e32864a-58aa-493a-b5b2-b29303cb103f" (UID: "7e32864a-58aa-493a-b5b2-b29303cb103f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:24:42.587721 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.587688 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-dshm" (OuterVolumeSpecName: "dshm") pod "7e32864a-58aa-493a-b5b2-b29303cb103f" (UID: "7e32864a-58aa-493a-b5b2-b29303cb103f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:24:42.587819 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.587736 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e32864a-58aa-493a-b5b2-b29303cb103f-kube-api-access-mds2t" (OuterVolumeSpecName: "kube-api-access-mds2t") pod "7e32864a-58aa-493a-b5b2-b29303cb103f" (UID: "7e32864a-58aa-493a-b5b2-b29303cb103f"). InnerVolumeSpecName "kube-api-access-mds2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 20:24:42.587819 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.587773 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e32864a-58aa-493a-b5b2-b29303cb103f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e32864a-58aa-493a-b5b2-b29303cb103f" (UID: "7e32864a-58aa-493a-b5b2-b29303cb103f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:24:42.686211 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.686171 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-model-cache\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:24:42.686211 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.686205 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mds2t\" (UniqueName: \"kubernetes.io/projected/7e32864a-58aa-493a-b5b2-b29303cb103f-kube-api-access-mds2t\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:24:42.686211 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.686216 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e32864a-58aa-493a-b5b2-b29303cb103f-tls-certs\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:24:42.686502 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.686225 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-dshm\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:24:42.686502 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.686237 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e32864a-58aa-493a-b5b2-b29303cb103f-home\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:24:42.955603 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.955511 2568 generic.go:356] "Generic (PLEG): container finished" podID="7e32864a-58aa-493a-b5b2-b29303cb103f" containerID="289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922" exitCode=0 Jul 14 20:24:42.955603 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.955550 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" Jul 14 20:24:42.955603 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.955560 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" event={"ID":"7e32864a-58aa-493a-b5b2-b29303cb103f","Type":"ContainerDied","Data":"289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922"} Jul 14 20:24:42.955603 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.955598 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59" event={"ID":"7e32864a-58aa-493a-b5b2-b29303cb103f","Type":"ContainerDied","Data":"f77e3f8bd99d3f9e6d8a831e73099acfe7038f6be7006c124869d0378b9acfb3"} Jul 14 20:24:42.955958 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.955618 2568 scope.go:117] "RemoveContainer" containerID="289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922" Jul 14 20:24:42.965797 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.965773 2568 scope.go:117] "RemoveContainer" containerID="289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922" Jul 14 20:24:42.966132 ip-10-0-128-126 kubenswrapper[2568]: E0714 20:24:42.966111 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922\": container with ID starting with 289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922 not found: ID does not exist" containerID="289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922" Jul 14 20:24:42.966205 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.966139 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922"} err="failed to get container status \"289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922\": rpc error: code = NotFound desc = could not find container \"289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922\": container with ID starting with 289c751e211c1b0a32d61b10c7362f031e708dbf554ec951257ceda885277922 not found: ID does not exist" Jul 14 20:24:42.977435 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.977394 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59"] Jul 14 20:24:42.980199 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:42.980174 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/auth-test-model-ecc78d-kserve-5d75fbd555-75n59"] Jul 14 20:24:43.046616 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:24:43.046580 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e32864a-58aa-493a-b5b2-b29303cb103f" path="/var/lib/kubelet/pods/7e32864a-58aa-493a-b5b2-b29303cb103f/volumes" Jul 14 20:25:16.936021 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.935983 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n"] Jul 14 20:25:16.936632 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.936402 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e32864a-58aa-493a-b5b2-b29303cb103f" containerName="main" Jul 14 20:25:16.936632 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.936419 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e32864a-58aa-493a-b5b2-b29303cb103f" containerName="main" Jul 14 20:25:16.936632 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.936496 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e32864a-58aa-493a-b5b2-b29303cb103f" containerName="main" Jul 14 20:25:16.939613 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.939590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:16.943677 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.943636 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"sub-test-model-ecc78d-kserve-self-signed-certs\"" Jul 14 20:25:16.943823 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.943724 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"default-dockercfg-gz8ps\"" Jul 14 20:25:16.943881 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.943857 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"kube-root-ca.crt\"" Jul 14 20:25:16.944038 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.944009 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"openshift-service-ca.crt\"" Jul 14 20:25:16.949732 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:16.949709 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n"] Jul 14 20:25:17.007851 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.007807 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdd395c-593e-4009-9a22-fb583477b996-tls-certs\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.007851 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.007853 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-home\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.008063 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.007918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj277\" (UniqueName: \"kubernetes.io/projected/4cdd395c-593e-4009-9a22-fb583477b996-kube-api-access-dj277\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.008063 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.007997 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-dshm\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.008063 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.008017 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-tmp-dir\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.008063 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.008044 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-model-cache\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.108990 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.108954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-model-cache\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.109176 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.109019 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdd395c-593e-4009-9a22-fb583477b996-tls-certs\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.109176 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.109046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-home\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.109176 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.109098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj277\" (UniqueName: \"kubernetes.io/projected/4cdd395c-593e-4009-9a22-fb583477b996-kube-api-access-dj277\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.109176 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.109136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-dshm\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.109176 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.109166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-tmp-dir\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.109486 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.109461 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-model-cache\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.109563 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.109503 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-home\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.109563 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.109532 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-tmp-dir\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.111497 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.111473 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-dshm\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.111792 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.111772 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdd395c-593e-4009-9a22-fb583477b996-tls-certs\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.119272 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.119250 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj277\" (UniqueName: \"kubernetes.io/projected/4cdd395c-593e-4009-9a22-fb583477b996-kube-api-access-dj277\") pod \"sub-test-model-ecc78d-kserve-79847bff47-nrt9n\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.251024 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.250928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:17.394263 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:17.394201 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n"] Jul 14 20:25:17.396608 ip-10-0-128-126 kubenswrapper[2568]: W0714 20:25:17.396577 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cdd395c_593e_4009_9a22_fb583477b996.slice/crio-6ed1fb443c85a11934e026a179bce55dc84ecfb98a6f245800e1e23009707883 WatchSource:0}: Error finding container 6ed1fb443c85a11934e026a179bce55dc84ecfb98a6f245800e1e23009707883: Status 404 returned error can't find the container with id 6ed1fb443c85a11934e026a179bce55dc84ecfb98a6f245800e1e23009707883 Jul 14 20:25:18.117390 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:18.117333 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" event={"ID":"4cdd395c-593e-4009-9a22-fb583477b996","Type":"ContainerStarted","Data":"56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37"} Jul 14 20:25:18.117840 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:18.117403 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" event={"ID":"4cdd395c-593e-4009-9a22-fb583477b996","Type":"ContainerStarted","Data":"6ed1fb443c85a11934e026a179bce55dc84ecfb98a6f245800e1e23009707883"} Jul 14 20:25:18.137885 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:18.137823 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" podStartSLOduration=2.137807279 podStartE2EDuration="2.137807279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:25:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:25:18.136664399 +0000 UTC m=+2841.610763145" watchObservedRunningTime="2026-07-14 20:25:18.137807279 +0000 UTC m=+2841.611906073" Jul 14 20:25:27.251476 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:27.251431 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:27.251476 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:27.251480 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:27.264028 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:27.264000 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:25:28.178849 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:25:28.178811 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:26:03.411025 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:03.410946 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n"] Jul 14 20:26:18.431619 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.431570 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" podUID="4cdd395c-593e-4009-9a22-fb583477b996" containerName="main" containerID="cri-o://56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37" gracePeriod=45 Jul 14 20:26:18.677656 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.677630 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:26:18.784244 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784138 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-dshm\") pod \"4cdd395c-593e-4009-9a22-fb583477b996\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " Jul 14 20:26:18.784244 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784199 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj277\" (UniqueName: \"kubernetes.io/projected/4cdd395c-593e-4009-9a22-fb583477b996-kube-api-access-dj277\") pod \"4cdd395c-593e-4009-9a22-fb583477b996\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " Jul 14 20:26:18.784244 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784224 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-home\") pod \"4cdd395c-593e-4009-9a22-fb583477b996\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " Jul 14 20:26:18.784244 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784253 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-tmp-dir\") pod \"4cdd395c-593e-4009-9a22-fb583477b996\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " Jul 14 20:26:18.784593 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784340 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdd395c-593e-4009-9a22-fb583477b996-tls-certs\") pod \"4cdd395c-593e-4009-9a22-fb583477b996\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " Jul 14 20:26:18.784593 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784398 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-model-cache\") pod \"4cdd395c-593e-4009-9a22-fb583477b996\" (UID: \"4cdd395c-593e-4009-9a22-fb583477b996\") " Jul 14 20:26:18.784593 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784551 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-home" (OuterVolumeSpecName: "home") pod "4cdd395c-593e-4009-9a22-fb583477b996" (UID: "4cdd395c-593e-4009-9a22-fb583477b996"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:26:18.784750 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784616 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4cdd395c-593e-4009-9a22-fb583477b996" (UID: "4cdd395c-593e-4009-9a22-fb583477b996"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:26:18.784816 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.784788 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-model-cache" (OuterVolumeSpecName: "model-cache") pod "4cdd395c-593e-4009-9a22-fb583477b996" (UID: "4cdd395c-593e-4009-9a22-fb583477b996"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:26:18.786437 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.786410 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-dshm" (OuterVolumeSpecName: "dshm") pod "4cdd395c-593e-4009-9a22-fb583477b996" (UID: "4cdd395c-593e-4009-9a22-fb583477b996"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:26:18.786437 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.786423 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cdd395c-593e-4009-9a22-fb583477b996-kube-api-access-dj277" (OuterVolumeSpecName: "kube-api-access-dj277") pod "4cdd395c-593e-4009-9a22-fb583477b996" (UID: "4cdd395c-593e-4009-9a22-fb583477b996"). InnerVolumeSpecName "kube-api-access-dj277". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 20:26:18.786579 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.786469 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cdd395c-593e-4009-9a22-fb583477b996-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4cdd395c-593e-4009-9a22-fb583477b996" (UID: "4cdd395c-593e-4009-9a22-fb583477b996"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:26:18.885446 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.885409 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-dshm\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:26:18.885446 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.885443 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj277\" (UniqueName: \"kubernetes.io/projected/4cdd395c-593e-4009-9a22-fb583477b996-kube-api-access-dj277\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:26:18.885446 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.885455 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-home\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:26:18.885683 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.885465 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-tmp-dir\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:26:18.885683 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.885477 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdd395c-593e-4009-9a22-fb583477b996-tls-certs\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:26:18.885683 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:18.885490 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cdd395c-593e-4009-9a22-fb583477b996-model-cache\") on node \"ip-10-0-128-126.ec2.internal\" DevicePath \"\"" Jul 14 20:26:19.412675 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.412590 2568 generic.go:356] "Generic (PLEG): container finished" podID="4cdd395c-593e-4009-9a22-fb583477b996" containerID="56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37" exitCode=0 Jul 14 20:26:19.412675 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.412634 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" Jul 14 20:26:19.412864 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.412636 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" event={"ID":"4cdd395c-593e-4009-9a22-fb583477b996","Type":"ContainerDied","Data":"56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37"} Jul 14 20:26:19.412864 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.412752 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n" event={"ID":"4cdd395c-593e-4009-9a22-fb583477b996","Type":"ContainerDied","Data":"6ed1fb443c85a11934e026a179bce55dc84ecfb98a6f245800e1e23009707883"} Jul 14 20:26:19.412864 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.412775 2568 scope.go:117] "RemoveContainer" containerID="56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37" Jul 14 20:26:19.421631 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.421608 2568 scope.go:117] "RemoveContainer" containerID="56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37" Jul 14 20:26:19.421910 ip-10-0-128-126 kubenswrapper[2568]: E0714 20:26:19.421886 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37\": container with ID starting with 56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37 not found: ID does not exist" containerID="56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37" Jul 14 20:26:19.421956 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.421919 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37"} err="failed to get container status \"56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37\": rpc error: code = NotFound desc = could not find container \"56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37\": container with ID starting with 56958efeea962bb04269c130dbb890f15e1445dd60de0ca3920d635b30607a37 not found: ID does not exist" Jul 14 20:26:19.430575 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.430534 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n"] Jul 14 20:26:19.435351 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:19.435316 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/sub-test-model-ecc78d-kserve-79847bff47-nrt9n"] Jul 14 20:26:21.046481 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:26:21.046443 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cdd395c-593e-4009-9a22-fb583477b996" path="/var/lib/kubelet/pods/4cdd395c-593e-4009-9a22-fb583477b996/volumes" Jul 14 20:27:57.230580 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:27:57.230546 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:27:57.238977 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:27:57.238952 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h7wbn_d07927f8-3b5c-4f1d-a3e8-ad78915b6c10/ovn-acl-logging/0.log" Jul 14 20:28:43.303152 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:43.303120 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-797bcdc8bf-d27v4_3a5843df-1ee6-4542-bd34-eeb629917806/authorino/0.log" Jul 14 20:28:49.083012 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:49.082978 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6546g_45e5c6ea-d802-4624-a83b-611e583d4206/server/0.log" Jul 14 20:28:49.344977 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:49.344875 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc_f112553b-066d-45af-892f-63185d600f0d/copy-manifests/0.log" Jul 14 20:28:49.352749 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:49.352720 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc_f112553b-066d-45af-892f-63185d600f0d/manager/0.log" Jul 14 20:28:53.315149 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:53.315089 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-797bcdc8bf-d27v4_3a5843df-1ee6-4542-bd34-eeb629917806/authorino/0.log" Jul 14 20:28:53.709673 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:53.709593 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d2h6c_25cb1ca7-4b33-46b5-b139-afaed6751dd3/kuadrant-console-plugin/0.log" Jul 14 20:28:54.872443 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:54.872407 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59f575c9d9-rzq6c_9efb5ff3-dca0-4a4b-9167-79b34afa5fcc/kube-auth-proxy/0.log" Jul 14 20:28:55.272272 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:55.272185 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-wbk6s_1eaa23e0-ca9a-4a2f-a787-c69ffd77a282/payload-pre-processing/0.log" Jul 14 20:28:55.526640 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:55.526556 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86f4bcd8c6-9lqpc_ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2/router/0.log" Jul 14 20:28:56.040565 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:56.040532 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-w4vj8_1bfdc529-c9f1-4cca-9c5b-513e578e7bc4/main/0.log" Jul 14 20:28:56.157182 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:56.157155 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jqzr9_5faa70f5-fafa-4375-a7c1-6a5f01b7c493/main/0.log" Jul 14 20:28:56.279273 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:56.279240 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbs7m7w_8c3c3496-16d3-4e6b-906f-a61e3d543fad/main/0.log" Jul 14 20:28:56.396237 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:56.396155 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-p5hps_c1b5b04a-fd1a-47c2-a157-1d0abfa48b5f/main/0.log" Jul 14 20:28:56.530047 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:28:56.530000 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-v77bp_937c5a94-eebe-4c07-82c3-4fee2a7f55bd/main/0.log" Jul 14 20:29:04.498557 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:04.498525 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rqk5f_df43b1ab-25e2-4928-80e8-91d3c6e80a59/global-pull-secret-syncer/0.log" Jul 14 20:29:04.642139 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:04.642105 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-krgkr_7ba863c7-4529-497a-a1b0-a4a178f229b7/konnectivity-agent/0.log" Jul 14 20:29:04.700426 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:04.700397 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-126.ec2.internal_443a9d45f6d9f6d7163dde949a4b961a/haproxy/0.log" Jul 14 20:29:09.743157 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:09.743118 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-797bcdc8bf-d27v4_3a5843df-1ee6-4542-bd34-eeb629917806/authorino/0.log" Jul 14 20:29:09.839626 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:09.839591 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d2h6c_25cb1ca7-4b33-46b5-b139-afaed6751dd3/kuadrant-console-plugin/0.log" Jul 14 20:29:11.785318 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:11.785276 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/alertmanager/0.log" Jul 14 20:29:11.829069 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:11.829039 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/config-reloader/0.log" Jul 14 20:29:11.884825 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:11.884796 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/kube-rbac-proxy-web/0.log" Jul 14 20:29:11.925849 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:11.925821 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/kube-rbac-proxy/0.log" Jul 14 20:29:11.964906 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:11.964874 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/kube-rbac-proxy-metric/0.log" Jul 14 20:29:12.012486 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:12.012458 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/prom-label-proxy/0.log" Jul 14 20:29:12.043731 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:12.043709 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a6a80ab-fe86-4c75-ba5d-3179cbd81298/init-config-reloader/0.log" Jul 14 20:29:12.461500 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:12.461400 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6q697_c5504cf7-d644-4f5a-9d2f-895717adfe7b/node-exporter/0.log" Jul 14 20:29:12.504308 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:12.504273 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6q697_c5504cf7-d644-4f5a-9d2f-895717adfe7b/kube-rbac-proxy/0.log" Jul 14 20:29:12.542395 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:12.542360 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6q697_c5504cf7-d644-4f5a-9d2f-895717adfe7b/init-textfile/0.log" Jul 14 20:29:13.056356 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:13.056329 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-lbhcq_43409727-caf9-4a21-bdf6-78605de73b61/prometheus-operator/0.log" Jul 14 20:29:13.085619 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:13.085588 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-lbhcq_43409727-caf9-4a21-bdf6-78605de73b61/kube-rbac-proxy/0.log" Jul 14 20:29:13.114239 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:13.114208 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-nn9rv_d2aeb254-67df-4d5b-b0b1-541bed0b2d41/prometheus-operator-admission-webhook/0.log" Jul 14 20:29:13.143977 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:13.143948 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5558b5987d-wbfrv_7ed6a649-6f09-465e-889c-c8979b96f242/telemeter-client/0.log" Jul 14 20:29:13.168149 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:13.168120 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5558b5987d-wbfrv_7ed6a649-6f09-465e-889c-c8979b96f242/reload/0.log" Jul 14 20:29:13.190940 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:13.190916 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5558b5987d-wbfrv_7ed6a649-6f09-465e-889c-c8979b96f242/kube-rbac-proxy/0.log" Jul 14 20:29:14.888322 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.888288 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4"] Jul 14 20:29:14.888806 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.888668 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cdd395c-593e-4009-9a22-fb583477b996" containerName="main" Jul 14 20:29:14.888806 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.888679 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cdd395c-593e-4009-9a22-fb583477b996" containerName="main" Jul 14 20:29:14.888806 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.888742 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cdd395c-593e-4009-9a22-fb583477b996" containerName="main" Jul 14 20:29:14.891672 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.891655 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:14.894553 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.894530 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7kv5j\"/\"default-dockercfg-7vshp\"" Jul 14 20:29:14.894692 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.894572 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7kv5j\"/\"kube-root-ca.crt\"" Jul 14 20:29:14.894692 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.894639 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7kv5j\"/\"openshift-service-ca.crt\"" Jul 14 20:29:14.901093 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.901065 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4"] Jul 14 20:29:14.988116 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.988083 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-podres\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:14.988326 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.988131 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-lib-modules\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:14.988326 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.988224 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm96r\" (UniqueName: \"kubernetes.io/projected/64f683b0-88f9-4cff-9d33-bfcf5125feff-kube-api-access-zm96r\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:14.988326 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.988261 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-proc\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:14.988494 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:14.988325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-sys\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089538 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089505 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-sys\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089708 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089560 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-podres\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089708 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089595 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-lib-modules\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089708 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089630 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-sys\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089708 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm96r\" (UniqueName: \"kubernetes.io/projected/64f683b0-88f9-4cff-9d33-bfcf5125feff-kube-api-access-zm96r\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089708 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089695 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-proc\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089884 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089735 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-podres\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089884 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089763 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-lib-modules\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.089884 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.089798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64f683b0-88f9-4cff-9d33-bfcf5125feff-proc\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.098070 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.098042 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm96r\" (UniqueName: \"kubernetes.io/projected/64f683b0-88f9-4cff-9d33-bfcf5125feff-kube-api-access-zm96r\") pod \"perf-node-gather-daemonset-hl4c4\" (UID: \"64f683b0-88f9-4cff-9d33-bfcf5125feff\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.202807 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.202723 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:15.334300 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:15.334268 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4"] Jul 14 20:29:15.336603 ip-10-0-128-126 kubenswrapper[2568]: W0714 20:29:15.336574 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod64f683b0_88f9_4cff_9d33_bfcf5125feff.slice/crio-d91420f5bc71565cce59f4ea399f989098446cf3e2e1cac23b7e86515f63c746 WatchSource:0}: Error finding container d91420f5bc71565cce59f4ea399f989098446cf3e2e1cac23b7e86515f63c746: Status 404 returned error can't find the container with id d91420f5bc71565cce59f4ea399f989098446cf3e2e1cac23b7e86515f63c746 Jul 14 20:29:16.236805 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:16.236767 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" event={"ID":"64f683b0-88f9-4cff-9d33-bfcf5125feff","Type":"ContainerStarted","Data":"2cf53b6f0a652d8587ad1741c6d1810164f1f5d018593799fcdf3a56219f096c"} Jul 14 20:29:16.237264 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:16.236831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" event={"ID":"64f683b0-88f9-4cff-9d33-bfcf5125feff","Type":"ContainerStarted","Data":"d91420f5bc71565cce59f4ea399f989098446cf3e2e1cac23b7e86515f63c746"} Jul 14 20:29:16.237264 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:16.236860 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:16.254281 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:16.254224 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" podStartSLOduration=2.254207201 podStartE2EDuration="2.254207201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:29:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:29:16.252406195 +0000 UTC m=+3079.726504925" watchObservedRunningTime="2026-07-14 20:29:16.254207201 +0000 UTC m=+3079.728305982" Jul 14 20:29:17.568583 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:17.568551 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zv28_8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f/dns/0.log" Jul 14 20:29:17.603335 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:17.603307 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zv28_8aa3cc3a-0d69-4e37-85c0-71358bc6fc3f/kube-rbac-proxy/0.log" Jul 14 20:29:17.763753 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:17.763725 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9pmc8_5bd55968-8301-4303-aa1a-f9853a39015d/dns-node-resolver/0.log" Jul 14 20:29:18.485429 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:18.485398 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xd4sq_c889a0ed-dd2e-4229-9fca-3417dfdd901a/node-ca/0.log" Jul 14 20:29:19.612187 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:19.612156 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59f575c9d9-rzq6c_9efb5ff3-dca0-4a4b-9167-79b34afa5fcc/kube-auth-proxy/0.log" Jul 14 20:29:19.754144 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:19.754100 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-wbk6s_1eaa23e0-ca9a-4a2f-a787-c69ffd77a282/payload-pre-processing/0.log" Jul 14 20:29:19.808972 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:19.808939 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86f4bcd8c6-9lqpc_ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2/router/0.log" Jul 14 20:29:20.558675 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:20.558643 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dzqfc_2137bf9d-7a42-4eb6-a328-3eb6418152ca/serve-healthcheck-canary/0.log" Jul 14 20:29:21.119593 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:21.119550 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7krf7_f8d0f259-c756-4852-af39-fd8dfd71282d/insights-operator/0.log" Jul 14 20:29:21.121755 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:21.121735 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7krf7_f8d0f259-c756-4852-af39-fd8dfd71282d/insights-operator/1.log" Jul 14 20:29:21.144868 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:21.144838 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bhclh_fa95c13a-6a35-48a6-8bc8-49b85d1ef299/kube-rbac-proxy/0.log" Jul 14 20:29:21.167894 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:21.167870 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bhclh_fa95c13a-6a35-48a6-8bc8-49b85d1ef299/exporter/0.log" Jul 14 20:29:21.191697 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:21.191673 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bhclh_fa95c13a-6a35-48a6-8bc8-49b85d1ef299/extractor/0.log" Jul 14 20:29:22.251604 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:22.251577 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-hl4c4" Jul 14 20:29:24.126513 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:24.126481 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6546g_45e5c6ea-d802-4624-a83b-611e583d4206/server/0.log" Jul 14 20:29:24.259251 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:24.259159 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc_f112553b-066d-45af-892f-63185d600f0d/manager/0.log" Jul 14 20:29:24.286433 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:24.286405 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fdfc6fcb8-7mlfc_f112553b-066d-45af-892f-63185d600f0d/copy-manifests/0.log" Jul 14 20:29:25.902952 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:25.902914 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f49b85c7f-cf8j6_42dc27a2-e553-4c08-9f76-3a34d295cffa/manager/0.log" Jul 14 20:29:25.933296 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:25.933270 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xmsb2_07f18ef7-a29b-4175-a223-c95e881863cf/openshift-lws-operator/0.log" Jul 14 20:29:26.266207 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:26.266178 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59f575c9d9-rzq6c_9efb5ff3-dca0-4a4b-9167-79b34afa5fcc/kube-auth-proxy/0.log" Jul 14 20:29:26.368962 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:26.368933 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-wbk6s_1eaa23e0-ca9a-4a2f-a787-c69ffd77a282/payload-pre-processing/0.log" Jul 14 20:29:26.406181 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:26.406149 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86f4bcd8c6-9lqpc_ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2/router/0.log" Jul 14 20:29:27.791119 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:27.791091 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59f575c9d9-rzq6c_9efb5ff3-dca0-4a4b-9167-79b34afa5fcc/kube-auth-proxy/0.log" Jul 14 20:29:27.891824 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:27.891794 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-wbk6s_1eaa23e0-ca9a-4a2f-a787-c69ffd77a282/payload-pre-processing/0.log" Jul 14 20:29:27.929156 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:27.929116 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86f4bcd8c6-9lqpc_ac28f7ae-17e2-4cde-8f9b-3a942d3ddbe2/router/0.log" Jul 14 20:29:31.620187 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:31.620102 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-dwjfl_d638838d-ea06-4909-8d9c-2c8509cb5f18/kube-storage-version-migrator-operator/1.log" Jul 14 20:29:31.621094 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:31.621072 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-dwjfl_d638838d-ea06-4909-8d9c-2c8509cb5f18/kube-storage-version-migrator-operator/0.log" Jul 14 20:29:32.708588 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:32.708559 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58jks_33d370ea-b86b-4e86-a173-0babca5f40e0/kube-multus-additional-cni-plugins/0.log" Jul 14 20:29:32.732190 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:32.732152 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58jks_33d370ea-b86b-4e86-a173-0babca5f40e0/egress-router-binary-copy/0.log" Jul 14 20:29:32.754741 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:32.754713 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58jks_33d370ea-b86b-4e86-a173-0babca5f40e0/cni-plugins/0.log" Jul 14 20:29:32.777100 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:32.777068 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58jks_33d370ea-b86b-4e86-a173-0babca5f40e0/bond-cni-plugin/0.log" Jul 14 20:29:32.800157 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:32.800130 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58jks_33d370ea-b86b-4e86-a173-0babca5f40e0/routeoverride-cni/0.log" Jul 14 20:29:32.823542 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:32.823517 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58jks_33d370ea-b86b-4e86-a173-0babca5f40e0/whereabouts-cni-bincopy/0.log" Jul 14 20:29:32.846715 ip-10-0-128-126 kubenswrapper[2568]: I0714 20:29:32.846687 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58jks_33d370ea-b86b-4e86-a173-0babca5f40e0/whereabouts-cni/0.log"